Benchmarks
Public protocols: what each benchmark tests, the capabilities and scenarios in scope, and the methodology fixed before results are published.
Released benchmarks
8 benchmarks, listed alphabetically. Published Result counts show what is public today; they are separate from test coverage and are not a ranking.
- Benchmark · Version 1
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?
- Capabilities
- 9
- Scenarios
- 26
- Participating tools
- 13
- Current published Results
- 22
- Benchmark · Version 1
AI Database Agents
Which AI database agent answers business questions about a live relational database correctly — in the company's own terms, and honestly when the data cannot answer?
- Capabilities
- 7
- Scenarios
- 28
- Participating tools
- 11
- Current published Results
- 16
- Benchmark · Version 1
Converting a complex PDF into clean Markdown with a hosted API
Which hosted API converts a complex, real-world PDF into faithful, usable Markdown?
- Capabilities
- 8
- Scenarios
- 12
- Participating tools
- 13
- Current published Results
- 16
- Benchmark · Version 1
Converting a complex PDF into clean Markdown with an open-source library
Which self-hostable open-source library converts a complex, real-world PDF — native text, dense tables, charts, images and scanned pages — into Markdown a reader can trust without opening the original?
- Capabilities
- 8
- Scenarios
- 12
- Participating tools
- 10
- Current published Results
- 0
- Benchmark · Version 1
Managed RAG
Which managed RAG service turns a folder of private documents into a query API that finds the right passage, answers only from what it retrieved, cites the source that actually supports the claim — and keeps up when the source changes?
- Capabilities
- 5
- Scenarios
- 15
- Participating tools
- 7
- Current published Results
- 1
- Benchmark · Version 1
Quiz Generation
Which AI quiz tool turns study material into valid quizzes, and then grades, explains and tracks what the learner actually understood?
- Capabilities
- 7
- Scenarios
- 15
- Participating tools
- 10
- Current published Results
- 5
- Benchmark · Version 1
Structured Document Extraction
Which document extraction platform turns a user-defined schema into correct structured records — across layouts, scans and repeated sets — and is honest about what it could not find?
- Capabilities
- 7
- Scenarios
- 19
- Participating tools
- 10
- Current published Results
- 30
- Benchmark · Version 1
Web Search for AI
Which web search or answer API finds the live-web material that actually supports an answer, keeps up when that material changes, and -- where it writes the answer itself -- says only what its own sources support?
- Capabilities
- 3
- Scenarios
- 10
- Participating tools
- 12
- Current published Results
- 0