By August 2017, lawyers from Asia, Europe, Canada and LATAM had contributed over 2,000 questions to a knowledge base in the open source Q&A Maker program, spanning the following topics: MOUs, NDAs, Sponsorship Agreements, Marketing, Privacy (including GDPR), Microsoft Cloud resources, compliance and company information (entities and signatories).

The core project team worked together to integrate Q&A Maker with Microsoft’s bot framework, including text analytics and Language Understanding Intelligence Service (LUIS) – which uses cognitive intelligence to derive the intent behind queries – and the Bing Translator API.

Improving client engagement

It quickly became apparent that the bot would need certain key features to incentivise broad use by CELA’s clients (and avoid Zela being relegated to the “just another tool” graveyard).  CELA lawyers and engineers worked together to develop bespoke code so that:

• Clients could either ask questions via free text field, or by selecting topics, sub-topics, and top 5 questions (which update automatically depending on user clicks and text queries)
• If a client asks a question in a way Zela doesn’t understand, or that is out of scope, Zela offers the client up to four options of questions within a lower confidence score, and gives the client the option to escalate to “Zela managers” (a team of paralegals that have visibility into a Zela portal and moderate feedback, escalations, train the bot, and loop in legal professionals from around the world if the matter requires urgent attention from a legal professional)
• Clients are asked whether Zela’s response was helpful – and the response is recorded (and monitored on an aggregated basis)
• The chat window had links to a privacy policy that explains how personal information will be used (to authenticate the user, or to respond to escalations) and that it will be deleted once it is no longer necessary
• Microsoft’s accessibility requirements were built in, so, for example, text can be auto-read (for people with sight impairments)
• the Q&A text could be translated for use in markets where clients don’t speak English in day-to-day tasks.

Measuring success

The CELA team also wanted to be able to measure Zela’s success, and how (and where) it was being used successfully.  So the team integrated Zela’s telemetry with Microsoft PowerBI, to collect data like:

• which subsidiaries are using Zela, and how often
• the most common questions, topics and sub-topics (i.e. is there something CELA should deliver training on)
• the amount of lawyer and paralegal time that has been shifted to higher impact work
• the number of times is Zela answering the client’s question, or suggesting topics
• the number of users, and questions
• client satisfaction with Zela interactions.

Rise of the Bots