Scenario definition · S39

The answer is available in the connected data

Whether the question can be answered from the connected data and the agent returns the correct result when it can.

What this scenario means

This scenario checks basic database-grounded answering. A good agent uses the connected data to answer when the information is present, rather than guessing, refusing, or drifting away from the question. It must return the right result from the available data and recognise when the database can support the answer.

What we evaluate

  • Whether the agent answers from the connected data when the information is available.
  • Whether the reply matches the correct result supported by the data.
  • Whether the agent stays grounded in the database rather than guessing or inventing an answer.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

Capabilities this scenario exercises

A scenario may exercise one or more capabilities.

Capability

Question Answering

Answers a question about the connected data correctly — finding the right data, using the company's meaning, matching the user's wording to what is stored — and says so when the data cannot answer.

Benchmarks that use this scenario

A scenario has global identity and may be reused across benchmarks.

The answer is available in the connected data — Scenario definition | AI Demos