Skip to main content

Improve incomplete or overlapping Answer training

Overview​

The Improvements View is divided into four tabs, Unanswered, Training Similarities, Not Helpful, and Clarifications.

  • The Unanswered tab provides you with a collection of questions that were asked by chatters that the bot did not understand, and responded to with the Not Understood Answer. In this view, you can also find the groupings of not understood Answers (topic clustering).

  • The Training Similarities tab enables you to quickly reduce the number of similar training questions between two distinct Answers.

  • The Not Helpful tab contains Answers that chatter reviewed as Not Helpful (reviewed with a thumbs down).

  • The Clarifications tab provides you with a collection of questions that were asked by chatters that the bot did not understand, and responded to with the Needs Clarification Answer.

To access your bot's Improvements View, click Improve in the left side panel.

Unanswered, Not Helpful, and Clarifications Tabs​

These views provide you with an opportunity to improve the chat experience for future chatters by training these questions to the appropriate Answers.

Below each question in the list, you are provided with:

  • Why the question appears on the list (ie. Not Understood, Needs Clarification, Not Helpful)

  • How long ago the question was asked

  • A link to view the conversation in its entirety

If there is an existing Answer within the bot that would be appropriate to provide for this question in the future, use the corresponding dropdown menu to select the answer, and then click Train.

You can also bulk train multiple questions to the same Answer. To do this, click to place a checkmark in the box next to each question you want to train to the same Answer. Use the dropdown menu provided at the bottom of the page to select the answer you want to train the questions to. Click Train.

Search questions​

Use the search field at the top to search for specific terms or phrases within the list of questions. This is especially useful when you have identified a trend amongst the questions and want to train all of the related questions to the same Answer.

Filter questions by variable (Unanswered tab only)​

Use the variable filters to sort unanswered questions based on a chatter's profile variables. Questions will be shown in a filter based on the first chatter to have asked the question. The results include up to 6 months of question history.

note

The variable filter looks at only the current values of a chatter's profile variables. It does not look at past values.

Here’s how to filter your Unanswered Questions page by variable:

  1. Select the Apply Filter button.

  2. Click the Select variable drop-down menu, then select the variable to filter for.

  3. Select the operator, then add the variable value to filter for in the Value field.

  4. Click And to add more variables to your filter.

  5. Click Apply.

Filtered results update every 20 minutes. Refresh the page to see the updated list.

Delete questions​

Click the trash can icon to delete a question from the list. It is recommended that you only delete questions that are gibberish and likely not to be asked by other chatters.

Training Insights​

Not Understood Groups​

Within the Unanswered Questions tab, the Training Insights feature groups similar Not Understood questions together. These groupings help you identify content gaps that could be filled with new Answers, as well as any gaps in the training for existing Answers. New groups are automatically populated once a day at midnight.

How to view Not Understood Groups​

  1. Go to the Improvements tab on the Ada dashboard.

  2. Select the Grouped view.

  3. Select a group in the Not Understood Groups column.

  4. From here, you can train one or more questions to a new or existing Answer

Interpreting Not Understood groups​

Each group under the Not Understood Groups heading is named with what Ada thinks is the common intent, or theme, of that group of questions. Inspect these intents to determine if they require attention.

  • For intents that your bot doesn’t have an answer for, consider building an Answer or a flow to address this.

  • If the questions in a Not Understood group should have been addressed, perhaps the training for that Answer or flow can be improved. If these Answers do have lots of training questions already, take a deeper look at the existing training to see if it is of high enough quality. For more information about training, see our Bot Training Overview.

  • If you aren’t sure what to make of it, ask your Ada team for guidance!

Additional Information​

  • New groups can’t be populated on-demand. They’re populated automatically once a day at midnight.

  • Groups only surface if there have been 150 Not Understood questions.

  • A maximum of 100 groups display at a time.

Training Similarities​

The Training Similarities tab identifies and prioritizes training questions that are similar and result in the bot providing the chatter with the Needs Clarification Answer. This feature has been designed to reduce the number of training questions in which the bot responds with Needs Clarification, and subsequently improve the chatter’s satisfaction with their bot experience.

To access the Training Similarities tab, click Improve > Training Similarities in the left side panel.

You will be provided with a prioritized list of training similarities that we encourage you to investigate. Addressing training similarities helps your bot know which answer to choose when a chatter asks a question, which leads to a higher clarification rate. The list is updated every 24 hours, and a timestamp is provided to indicate when it was last updated.

Each row identifies a pair of Answers that have been flagged as similar, leading the bot to have to ask the chatter to clarify their intent. Below the Answer titles you are provided with the number of training question similarities that have been identified, as well as the total number of times those similarities have resulted in the Needs Clarification answer being provided to chatters.

Clarification Rate Report​

On the right hand side of the page you can click View Clarification Rate Report to be taken to the Clarification Rate Report which shows the percent of conversations in which the bot responded with the Needs Clarification answer at least once. Resolving the indicated training similarities should result in a reduction in the clarification rate, and subsequently an increase in chatter satisfaction.

Resolve Training Similarities​

Click on the cell that contains the training similarities you’d like to evaluate. This takes you to the Resolve Training Similarities page.

At the top of each of the two columns is the name of an Answer within your bot. Each row below contains pairs of training questions that are similar to each other, and when asked by a chatter, the bot provides the Needs Clarification answer.

This view allows you to quickly identify which training questions are similar across a pair of Answers and decide where the training question should be best situated in an effort to decrease the frequency of the bot providing the Needs Clarification answer to chatters.

To review the Answer’s content before determining where training questions are best suited, hover over the answer’s title to reveal the expand icon, and click on it to open the answer in the Answer View.

There are a number of different actions you can take:

  • Move Training If a training question is trained to an answer, but it is more applicable to the other answer, click the arrow to move the training question from one answer to another. Do this to as many training questions as you deem necessary. Click Update to set the revised training.

  • Edit Training Click the down arrow on the far right of an answer’s cell to view a menu of possible actions. Click Edit Training to be provided with a text box allowing you to edit the training question. Click Save to set this revised training.

  • Delete Training Click the down arrow on the far right of an answer’s cell to view a menu of possible actions. Clicking Delete Training will prompt you to verify that you want to delete the training question from the answer. Click Delete to confirm.

  • Move all to.... To move all of the similar training questions to one Answer, click the Move all to… dropdown in the upper right corner, and select the Answer you want all of the training questions to be moved to. Click Update to set the revised training.

Have any questions? Contact your Ada team—or email us at .