When your chatbot has multiple AI Agents, our AI Supervisor automatically
assigns the most appropriate Agent to handle the user’s input queries. It does so by
first determining the user’s intent, then looks for the most
suitable Agent to help the user based on Agents’ names and descriptions.
In our article Multi-Agent Chatbot, we covered some best practices on
Agent description writing and provided a number of examples. However, as comprehensive as the
AI Supervisor tries to be, there will still be scenarios where Agent descriptions do not sufficiently
cover all scenarios.
When AI Agents are created or modified, we automatically generate a number of intents along
with example user queries to serve as reference for the AI Supervisor. These intents can be reviewed,
customized, or deleted through the AI Supervisor Settings interface. The AI Supervisor makes use
of these intents directly when evaluating each Agent’s specialty.
To fine-tune Agent intents, go to AI Agents tab on the left navigation menu, then
click on AI Supervisor Settings:
Now head to the “Intents” tab:
Here, you will see the intents that have been automatically generated by the AI Supervisor based on
its interpretation of the Agent’s function. When a user submits a query, if you have more than one active
AI Agents set up, then the AI Supervisor will extract the user’s intent and try to find the closest match
across all active Agents.
To add new intents, select the Agent that you want to fine-tune, then click “Add intent”.
Then, specify the intent along with a few example queries that users could input which demonstrates this intent.
Click Create.
You should see the intent added under “Fixed Intents”. You may need to scroll down to see the entire list.
You can also edit AI-Generated Intents. Simply open the three-dot-menu next to the intent and select “Edit”.
After you save your edits, this intent will be moved to the “Fixed Intents” section.
Note: AI-Generated Intents will be overwritten each time you edit the Agent
name, description, or prompt. On the other hand, Fixed Intents are permanent
and only deleted if you manually do so.
When should I fine-tune Agent intents?
For most use cases involving general Q&A, fine-tuning intents is not necessary. Our own internal tests have
shown that AI-generated intents are typically sufficient for good Agent routing.
Allowing bot makers to control intent classification at a granular level empowers them to enhance chatbot performance
in situations where many AI Agents are involved, or where the chatbot requires a high degree of precision.
When designing a multi-agent setup, we recommend keeping to a “MECE” approach.
“MECE” refers to - Mutually Exclusive, Collectively Exhaustive. In other words, no gaps, no overlaps.
Ideally, you want the situation where:
User-facing Agent A handles:
- intent class 1
- intent class 2
- intent class 3
- etc.
User-facing Agent B handles:
- intent class 10
- intent class 11
- intent class 12
- etc.
and there is minimal overlap in intents among Agents.
All your Agents combined should cover all possible intents your users might have.
But since you can’t ever account for all intents in the universe, and given the fact that intents are
presented in natural language which can have multiple permutations to express the same meaning,
there is no deterministic way to get 100% consistency. But you can get pretty close with an
optimized setup.
During your chatbot’s operations, it is important to monitor incoming conversations and identify instances
where the wrong Agent was chosen to handle a user’s query. In these situations, we recommend that you make
use of our “Debug mode”. In this mode, if you have multiple user-facing Agents set up, you will see “Intent”
and “Active Agent” show up alongside every user query that underwent Agent routing.
If the incorrect Agent has been assigned, a quick and effective fix is to add the intent and the corresponding
user query to what should have been the correct Agent to handle the request within AI Supervisor Settings -> Intents.
This will ensure that if queries of the same type were to come up again, the correct Agent would be chosen.
Providing more examples for this intent will further reinforce the ability for the AI Supervisor to classify
similar cases in the future.