Scenario definition · S13
Retrieve information from an external system
A request requires information to be retrieved from an external connected system.
What this scenario means
This scenario covers a support interaction where the correct response depends on looking up live information in an external system. The expected behaviour is to use that connected source, then answer with the retrieved information in a way that reflects the current record.
What we evaluate
- Whether the response uses an external connected system to obtain the needed information.
- Whether the answer reflects the information returned by that system.
- Whether the agent does not rely on the knowledge base alone when the needed information lives elsewhere.
◌Test-case detail. Specific test-case inputs and grading live on the related test-case pages.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Action execution
Looks things up and makes changes in connected systems (orders, subscriptions, tickets)
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?