Converting a complex PDF into clean Markdown with a hosted API
This benchmark helps buyers choose a hosted API for PDF-to-Markdown conversion.
Benchmark overview
What is included and excluded
Evaluates whether a hosted API turns complex PDFs into faithful, usable Markdown without dropping, reordering, or inventing content.
A reader learns whether the API preserves the document's text, order, structure, tables, figures, scanned pages, equations, and code, and whether it handles failures honestly when the source cannot be represented cleanly.
In scope
- Conversion of complex, real-world PDFs into faithful Markdown.
- Preservation of text, reading order, headings, section structure, tables, figures, charts, scanned text, equations, and code.
- Honest fallback when a source element cannot be expressed cleanly in Markdown.
Out of scope
- Structured field extraction.
- Web-page conversion.
- Form filling.
- Handwriting.
- Document classification and routing.
- PDF creation or editing.
Participating tools
Tools in this benchmark’s public roster. Publication availability is not a performance ranking.
| Tool | Published Results | Explore |
|---|---|---|
| Adobe PDF Extract API | 1 scenario with a published Result | View tool in this benchmark → |
| Extend AI | 1 scenario with a published Result | View tool in this benchmark → |
| Mistral AI | 9 scenarios with published Results | View tool in this benchmark → |
| Reducto | 5 scenarios with published Results | View tool in this benchmark → |
| Firecrawl | No published result | View tool in this benchmark → |
| GPT-5.6 terra | No published result | View tool in this benchmark → |
| Landing AI | No published result | View tool in this benchmark → |
| LlamaParse | No published result | View tool in this benchmark → |
| Nutrient.io | No published result | View tool in this benchmark → |
| PDF Vector | No published result | View tool in this benchmark → |
| Tensorlake | No published result | View tool in this benchmark → |
| Unstructured | No published result | View tool in this benchmark → |
| Upstage AI | No published result | View tool in this benchmark → |
Capabilities & scenarios
12 scenarios grouped by 8 capabilities. Open a group to explore its scenarios in this benchmark.
Text Fidelity1 scenario
Ordinary digital text survives completely and correctly in the Markdown output.
Capability in this benchmark → · Global definition →
- An ordinary digital text document1 tool with a published Result
Reading Order & Layout1 scenario
The source reading sequence survives the spatial-to-linear conversion without the columns being interleaved or reordered.
Capability in this benchmark → · Global definition →
- A page laid out in multiple columns1 tool with a published Result
Heading & Section Structure2 scenarios
Document structure survives as real Markdown headings and sections, with nesting and footnotes kept in the right relationship.
Capability in this benchmark → · Global definition →
- A document with styled headings and subheadings1 tool with a published Result
- Footnotes at the bottom of the page1 tool with a published Result
Table Extraction2 scenarios
Rows, columns, headers, and values survive as a table with the right relationships intact.
Capability in this benchmark → · Global definition →
- A simple, clearly formatted table1 tool with a published Result
- A table that continues across a page break1 tool with a published Result
Figures & Charts2 scenarios
Visual content survives with its position, caption or title, and a text-reachable trace for later use.
Capability in this benchmark → · Global definition →
- A document with figures and captions1 tool with a published Result
- A document with a data chartNo published results
Scanned Document OCR2 scenarios
Text that exists only as pixels is recovered into usable output instead of being silently lost.
Capability in this benchmark → · Global definition →
- A cleanly scanned document3 tools with published Results
- A document mixing digital and scanned pages2 tools with published Results
Equations & Mathematical Notation1 scenario
Mathematical notation survives as math markup or an honest fallback, without being turned into misleading prose.
Capability in this benchmark → · Global definition →
- A document containing mathematical equations2 tools with published Results
Code Extraction1 scenario
Code survives as a preformatted fenced block with its whitespace and line structure intact.
Capability in this benchmark → · Global definition →
- A document containing a code block2 tools with published Results
Results overview
Current published evidence in this benchmark.
Publication availability is separate from test coverage and unpublished research progress.
How the benchmark works
A public summary of the evaluation method. The same defined scope and evidence standard apply to every tool assessed under this version.
One test case, one tool
One test case is scored for one tool at a time.
Pass, fail, or partial
Each run ends pass, fail, or partial. A missing capability scores 0 and stays in the arithmetic — a product must not rank higher by having less product. not_applicable, not_measured and blocked (with the reason, never a 0) are distinct states.
Versioned scenarios and resources
Use versioned scenarios and registered resource versions so runs stay comparable.
Recorded stimulus and complete output
Record the input stimulus, the complete output, and the relevant registered references.
Resources and fixtures
The registered material and systems that create a consistent test environment for this benchmark.
PDF→Markdown fixture corpus — round 1
A shared pinned PDF corpus used as input for both PDF-to-Markdown benchmark versions.