.png){kind=icon}
There is a lot of information on the government websites, FAQs, documents, etc. Standard FAQ websites are a great start, but it can sometimes be difficult for a person to find the information that they need. Even standard web site search boxes are inefficient. By using a bot backed with AI citizens will more quickly find the relevant information they require.
What if a bot greeted them when they arrived and ask them what they needed? We could create a FAQ for them to integrate with the bot so that users could ask questions and figure out what they need and route to an email or helpdesk if the question couldn’t be answered.
QnA informational bots enable customers to disseminate information about their organizations to external or internal audiences at scale. Bots provide an additional channel for your customers to access information. Promoting a custom bot as the first place to go to get answers about your organization and its programs, reduces the number of inquiries that your customer service organization must handle. A typical informational bot can be developed in less than a day.
Risks
Make sure you have the right data and someone who can further refine the QnA bot through Active Learning.
Rationale
As a channel of access to any government service, implementation of chatbots in various government agencies will transform the communication between citizens and government, providing benefits like consistent, quick and reliable information and improve citizen satisfaction. If they cannot find the answer through the chatbot, it can route to a help desk or email request, saving time, making more efficient use of resources and lowering costs of service.
As-Is
|
QnA Maker is a cloud-based Natural Language Processing (NLP) service that allows you to create a natural conversational layer over your data. It is used to find the most appropriate answer for any input from your custom knowledge base (KB) of information. QnA Maker is commonly used to build conversational client applications, which include social media applications, chat bots, and speech-enabled desktop applications. QnA Maker doesn't store customer data. All customer data (question answers and chat logs) is stored in the region the customer deploys the dependent service instances in. QnA Maker imports your content into a knowledge base of question and answer pairs. The import process extracts information about the relationship between the parts of your structured and semi-structured content to imply relationships between the question and answer pairs. You can edit these question and answer pairs or add new pairs. The content of the question and answer pair includes:
After you publish your knowledge base, a client application sends a user's question to your endpoint. Your QnA Maker service processes the question and responds with the best answer. Create a chat bot programmatically Once a QnA Maker knowledge base is published, a client application sends a question to your knowledge base endpoint and receives the results as a JSON response. A common client application for QnA Maker is a chat bot. Build low code chat bots The QnA Maker portal provides the complete knowledge base authoring experience. You can import documents, in their current form, to your knowledge base. These documents (such as an FAQ, product manual, spreadsheet, or web page) are converted into question and answer pairs. Each pair is analyzed for follow-up prompts and connected to other pairs. The final markdown format supports rich presentation including images and links. Once your knowledge base is edited, publish the knowledge base to a working Azure Web App bot without writing any code. Test your bot in the Azure portal or download it and continue development. High quality responses with layered ranking QnA Maker's system is a layered ranking approach. The data is stored in Azure search, which also serves as the first ranking layer. The top results from Azure search are then passed through QnA Maker's NLP re-ranking model to produce the final results and confidence score. Multi-turn conversations QnA Maker provides multi-turn prompts and active learning to help you improve your basic question and answer pairs. Multi-turn prompts give you the opportunity to connect question and answer pairs. This connection allows the client application to provide a top answer and provides more questions to refine the search for a final answer. After the knowledge base receives questions from users at the published endpoint, QnA Maker applies active learning to these real-world questions to suggest changes to your knowledge base to improve the quality. Development lifecycle QnA Maker provides authoring, training, and publishing along with collaboration permissions to integrate into the full development life cycle. |
To-Be
Custom question answering, a feature of Azure Cognitive Service for Language was introduced in May 2021 with several new capabilities including enhanced relevance using a deep learning ranker, precise answers, and end-to-end region support. Each custom question answering project is equivalent to a knowledge base in QnA Maker. You can easily migrate knowledge bases from a QnA Maker resource to custom question answering projects within a language resource. You can also choose to migrate knowledge bases from multiple QnA Maker resources to a specific language resource.
To successfully migrate knowledge bases, the account performing the migration needs contributor access to the selected QnA Maker and language resource. When a knowledge base is migrated, the following details are copied to the new custom question answering project:
- QnA pairs including active learning suggestions.
- Synonyms and default answer from the QnA Maker resource.
- Knowledge base name is copied to project description field.
Resource level settings such as Role-based access control (RBAC) are not migrated to the new resource. These resource level settings would have to be reconfigured for the language resource post migration. You will also need to re-enable analytics for the language resource.
Constraints
Avoid high-risk scenarios: The machine learnt algorithm used by QnA Maker optimizes the performance based on the data it is trained on, however there will always be edge cases where the correct answer isn't returned for a user query which the system doesn't understand well. When you design your scenarios with QnA Maker, be aware of the possibility of false positive results. It is advisable to create a dataset of the top queries asked in your scenario and the corresponding expected answers, and periodically test the service for the correctness of the responses. For example:
Avoid open domain scenarios: QnA Maker is meant to answer questions from a particular domain knowledge base, not open-ended questions, or out-of-domain questions. Using out-of-domain questions with QnA Maker runs the risk of returning incorrect responses.
Transformation Enablers
These are solutions where a simple QnA Maker bot will yield value:
- When you have static information - Use QnA Maker when you have static information in your knowledge base of answers. This knowledge base is custom to your needs, which you've built with documents such as PDFs and URLs.
- When you want to provide the same answer to a request, question, or command - when different users submit the same question, the same answer is returned.
- When you want to filter static information based on meta-information - add metadata tags to provide additional filtering options relevant to your client application's users and the information. Common metadata information includes chit-chat, content type or format, content purpose, and content freshness.
- When you want to manage a bot conversation that includes static information - your knowledge base takes a user's conversational text or command and answers it. If the answer is part of a pre-determined conversation flow, represented in your knowledge base with multi-turn context, the bot can easily provide this flow
Stakeholders
Azure Administrator, Customer Service Representatives
Business Process Model
Before you can create any QnA Maker knowledge bases, you must first set up a QnA Maker service in Azure. Anyone with authorization to create new resources in a subscription can set up a QnA Maker service. If you are trying the Custom question answering feature, you would need to create the Language resource and add the Custom question answering feature.
https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/
Lessons Learned
Below are some best practices: Best practices - QnA Maker - Azure Cognitive Services | Microsoft Docs
Extraction
The QnA Maker service is continually improving the algorithms that extract QnAs from content and expanding the list of supported file and HTML formats. Follow the guidelines for data extraction based on your document type.
In general, FAQ pages should be stand-alone and not combined with other information. Product manuals should have clear headings and preferably an index page.
Configuring multi-turn
Create your knowledge base with multi-turn extraction enabled. If your knowledge base does or should support question hierarchy, this hierarchy can be extracted from the document or created after the document is extracted.
Creating good questions and answers
Good questions
The best questions are simple. Consider the key word or phrase for each question then create a simple question for that key word or phrase.
Add as many alternate questions as you need but keep the alterations simple. Adding more words or phrasings that are not part of the main goal of the question does not help QnA Maker find a match.
Add relevant alternative questions
Your user may enter questions with either a conversational style of text, How do I add a toner cartridge to my printer? or a keyword search such as toner cartridge. The knowledge base should have both styles of questions in order to correctly return the best answer. If you aren't sure what keywords a customer is entering, use Application Insights data to analyze queries.
Good answers
The best answers are simple answers but not too simple. Do not use answers such as yes and no. If your answer should link to other sources or provide a rich experience with media and links, use metadata tagging to distinguish between answers, then submit the query with metadata tags in the strictFilters property to get the correct answer version.
Ranking/Scoring
Make sure you are making the best use of the ranking features QnA Maker supports. Doing so will improve the likelihood that a given user query is answered with an appropriate response.
Add alternate questions
Alternate questions improve the likelihood of a match with a user query. Alternate questions are useful when there are multiple ways in which the same question may be asked. This can include changes in the sentence structure and word-style.
Use metadata tags to filter questions and answers
Metadata adds the ability for a client application to know it should not take all answers but instead to narrow down the results of a user query based on metadata tags. The knowledge base answer can differ based on the metadata tag, even if the query is the same. For example, "where is parking located" can have a different answer if the location of the restaurant branch is different - that is, the metadata is Location: Seattle versus Location: Redmond.
Use synonyms
While there is some support for synonyms in the English language, use case-insensitive word alterations via the Alterations API to add synonyms to keywords that take different forms. Synonyms are added at the QnA Maker service-level and shared by all knowledge bases in the service.
Use distinct words to differentiate questions
QnA Maker's ranking algorithm, that matches a user query with a question in the knowledge base, works best if each question addresses a different need. Repetition of the same word set between questions reduces the likelihood that the right answer is chosen for a given user query with those words.
Collaborate
QnA Maker allows users to collaborate on a knowledge base. Users need access to the Azure QnA Maker resource group in order to access the knowledge bases. Some organizations may want to outsource the knowledge base editing and maintenance, and still be able to protect access to their Azure resources. This editor-approver model is done by setting up two identical QnA Maker services in different subscriptions and selecting one for the edit-testing cycle. Once testing is finished, the knowledge base contents are transferred with an import-export process to the QnA Maker service of the approver that will finally publish the knowledge base and update the endpoint.
Active learning
Active learning does the best job of suggesting alternative questions when it has a wide range of quality and quantity of user-based queries. It is important to allow client-applications' user queries to participate in the active learning feedback loop without censorship. Once questions are suggested in the QnA Maker portal, you can filter by suggestions then review and accept or reject those suggestions.