Scenario definition · S19
Customer switches languages during the conversation
A conversation switches languages mid-stream and must continue without losing context.
What this scenario means
This scenario tests whether the agent can keep the thread of the conversation when the language changes during an ongoing exchange. It is hard because the agent has to carry forward earlier context, understand the new-language turn, and keep responding coherently without forcing a restart or dropping the topic.
What we evaluate
- Whether the agent preserves earlier context after the language change.
- Whether the agent continues the exchange in the new language.
- Whether the agent keeps the topic and prior thread intact rather than restarting the conversation.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Multilingual support
Handles customers who don't write in English
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?