The request uses different wording than the source
A request uses wording different from the source, but the system must still resolve the same underlying fact or passage.
What this scenario means
This scenario tests whether wording differences break matching. The request and the source do not use the same words, but the system should still recognise the same meaning and resolve the correct underlying fact or passage. A correct response shows that paraphrase does not prevent the system from finding the intended source.
What we evaluate
- Whether the system recognises the same underlying fact or passage despite different wording.
- Whether it relies on meaning rather than an exact phrase match.
- Whether it still resolves the correct source when the request is paraphrased.
- Whether it treats different wording as equivalent when the underlying meaning is the same.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Knowledge-grounded answering
Answers customer questions from the company's own knowledge base — not from what the model assumes
Retrieval
The passage that answers the question comes back when the documents hold it — and the absence is discoverable when they don't. Judged against the documents: did the supporting passage come back?
Web Retrieval
When the web holds material that supports the answer, does it come back? Judged against the WEB, not against what the tool retrieved: if the answer was out there and search missed it, that is a retrieval failure. The developer does not know which URL holds the answer -- the product has to find it, which is the whole category.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Managed RAG
Also uses this scenario.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?
Web Search for AI
Also uses this scenario.