Can emit the bank-statement extraction as nested schema-shaped JSON, with separate metadata, account_holder/account, branch, transactions, summary, rewards, and disclaimers objects instead of flat OCR text.
What was measured
Schema Adherence
Does the output follow the supplied JSON schema hierarchy exactly, with correct nesting, field names, and data types?
decisive for this rankingtransformation
If the tool does not follow the supplied JSON schema exactly, the extracted data cannot be reliably used for structured querying or downstream automation. (3 of 3 judges)
What was given, what came back
Test input: Bank Statement PDF · pdf · group: business-document-extraction
Input — what we sent
Research media bank statement 2 jul.png
Bank Statement PDF
A four-page bank statement PDF with dense transaction tables, balance-forward bridges, account metadata, rewards data, and disclaimer text. It was used to stress schema-driven extraction, multi-page continuity, row completeness, and financial numerical accuracy.
Why this input is hard
- · Table extraction across 50+ transaction rows
- · Multi-page continuity with balance-forward bridges
- · Parsing structured account metadata alongside unstructured transaction descriptions
- · Numerical accuracy for deposits, withdrawals, running balances, and summaries
- · Extraction of nested rewards and disclaimer sections
Output — unretouched
Loading file...
Also checked on this input — same tool, 5 other criteria
Extraction Accuracy⚠ StruggledMisplaces a bank-statement transaction across dates: a row that belongs to 18 Jun is attached to the 19 Jun record after merging, so date association is unreliable.Semantic Field Enrichment◐ MixedClassifies bank-statement transaction_type inconsistently on merged rows: the 28 Jun record is labeled Deposit even though the visible row shows only a 399 withdrawal amount, and another merged row is labeled Deposit/Withdrawal.Semantic Field Enrichment✗ FailedLeaves the derived bank-statement transaction_id field null, including on the 18 Jun ATM withdrawal that visibly contains the embedded ID 916910098754.Table & Record Completeness⚠ StruggledMerges adjacent bank-statement rows into one record; the 21 Jun crop shows two separate transactions, but the extracted description collapses them into one long UPI/merchant string.Table & Record Completeness✗ FailedOver-segments the bank-statement table: the report says 54 transaction records were extracted where 51 were expected, a +3 overcount caused by continuation rows and split records.
Provenance
- Observation
- 62e6efd4-2d0b-4870-8159-8b09877da655
- Evidence run
- a061b9e7-a9c5-443d-a171-b296aaf51b8c
- Study
- Extract and query structured data from documents using natural language
- Research task
- 86b9y25e5
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "datalab",
scenario: "business-document-extraction"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 7 other tools
measured on Schema Adherence
Extend AI✓ WorkedThe tool reconstructs the bank-statement hierarchy into nested JSON with branch, account, rewards, metadata, balances, summary, and transaction-related objects present in the requested layout.Landing AI✓ WorkedReconstructs the supplied bank-statement hierarchy instead of flat OCR, with nested statement.metadata, account_holder.address, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers objects present in the extracted JSON flow.LlamaParse✓ WorkedThe bank-statement output follows the requested nested schema closely, reconstructing metadata, account_holder, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers as structured objects rather than flat OCR text.Nanonets✓ WorkedPreserves the supplied bank-statement hierarchy in structured JSON, with nested statement.metadata, account_holder, account, balances, transactions, summary, rewards, and disclaimers objects instead of flattening the document into OCR text.Reducto✓ WorkedPreserves a nested, schema-shaped JSON structure for the bank statement instead of flattening the document into raw OCR, including top-level objects like metadata, account_holder, account, branch, statement_period, transactions, and summary.Retab✓ WorkedReconstructs the supplied bank-statement schema into a nested JSON object with separate statement.metadata, account_holder, account, branch, balances, transactions, summary, rewards, and disclaimers sections instead of flattening the document into OCR text.Unstract✓ WorkedFollows the requested nested bank-statement schema instead of flattening the document, populating structured objects such as metadata, account_holder, account, branch, balances, transactions, and summary.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com