Scenario definition · S12
Agent cannot resolve the request
A request that the agent cannot complete from the available tools, endpoints, or policy and must decline or hand off.
What this scenario means
This scenario covers requests that cannot be resolved within the available tools, endpoints, or policy. The correct behaviour is to acknowledge that the request cannot be completed and to provide the appropriate route or handoff instead of fabricating an action or answer.
What we evaluate
- Whether the request is recognised as unsupported by the available tools, endpoints, or policy.
- Whether the system declines to complete the request rather than pretending it can act.
- Whether the system provides an appropriate handoff or next-step route.
◌Test-case detail. Specific test-case inputs and grading live on the related test-case pages.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Escalation and human handoff
Knows when a conversation should go to a human, and hands it over properly
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?