Scenario definition · S11
A configured rule requires escalation
A configured rule requires escalation to a human.
What this scenario means
This scenario checks whether a configured rule or threshold triggers mandatory escalation. The correct behaviour is a human handoff when the configured condition is met, rather than the product handling the case entirely itself.
What we evaluate
- Whether the product recognises that the configured rule requires escalation.
- Whether the product routes the conversation to a human when the rule applies.
- Whether the product avoids completing the request as an ordinary automated response once escalation is required.
◌Test-case detail. Specific test-case inputs and grading live on the related test-case pages.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Escalation and human handoff
Knows when a conversation should go to a human, and hands it over properly
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?