The database's names are cryptic and the user documents them
A user documents cryptic table or column names so later questions can be interpreted correctly.
What this scenario means
This scenario tests whether the agent can use user-provided documentation to resolve unreadable database names and reach the right data. A good agent treats those descriptions as interpretive guidance, not as decoration, and applies them when answering later questions. It should use the documented meaning rather than guessing from the raw table or column name.
What we evaluate
- Whether the agent uses the supplied documentation when resolving a later question about the cryptic names.
- Whether the answer is grounded in the documented table or column rather than in a guess from the raw name.
- Whether the same documentation still works when the later question is phrased differently.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Training
Whether supplying company-specific information measurably improves future behaviour and generalises beyond the example given.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Database Agents
Which AI database agent answers business questions about a live relational database correctly — in the company's own terms, and honestly when the data cannot answer?