The user narrows what they just asked
A follow-up adds a narrowing constraint to an earlier question instead of replacing it.
What this scenario means
This tests whether the agent keeps conversational context when a request becomes more specific. A good agent treats the follow-up as a refinement of the same question, preserves the earlier intent, and applies the new constraint to that same line of inquiry rather than restarting or switching topics.
What we evaluate
- Whether the agent preserves the earlier question’s subject when answering the follow-up.
- Whether the agent applies the added constraint to the existing request rather than replacing the request.
- Whether the agent avoids silently broadening, shifting, or resetting the question.
- Whether the reply reflects the narrower scope only, not the earlier broader scope alone.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Conversational Interaction
Holds a working session rather than a series of isolated queries: context carried, questions refined, direction changed, ambiguity raised.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Database Agents
Which AI database agent answers business questions about a live relational database correctly — in the company's own terms, and honestly when the data cannot answer?