Schema Design for Document Extraction: The Hour That Decides Your Accuracy

The schema is the real prompt

Modern extraction runs against a structured-output contract: you hand the model a document and a shape, and it fills the shape. That shape does far more work than most teams realise. Field names carry semantics, types constrain the answer space, enums remove judgement calls, and required-versus-optional tells the model whether absence is an acceptable outcome. A model reading amount has to guess whether you want the net, the gross, or the amount actually payable after withholding. A model reading gross_amount has one target.

This is why schema work outperforms prompt work. A prompt instruction like “return the total including taxes” lives in a paragraph the model weighs against everything else in context. A field named gross_amount with a numeric type and a sibling called net_amountis a structural constraint: the ambiguity is gone before inference starts. Every hour spent disambiguating the shape removes a class of errors permanently, while prompt patches tend to fix one document and break another.

Model the document, not your database

The most common structural mistake is writing the extraction schema as a mirror of the target table in the ERP. It seems efficient — one less mapping layer — but it forces the model to do two jobs at once: read the document and transform its contents into your internal representation. Those jobs have different failure modes, and merging them makes failure impossible to attribute. When the cost centre comes back wrong, you cannot tell whether the model misread the page or misapplied your allocation rule.

Keep the extraction schema faithful to what is printed on the page, and do the business transformation in deterministic code afterwards. If the invoice shows a supplier name and a tax ID, extract exactly those; resolve them to your vendor ID in a lookup you can test, audit and fix without re-running extraction. The rule of thumb: if a value cannot be pointed at with a finger on the page, it does not belong in the extraction schema.

Five type decisions that cause most of the pain

Numbers are numbers

Returning “R$ 42.810,00” as a string pushes parsing into whatever code consumes the JSON, where locale rules get reinvented badly. Brazilian documents use a comma decimal separator and a dot for thousands, so a naive parser reading that string produces 42.81 — a thousandfold error that will not look obviously wrong in a report. Declare monetary fields as numbers, normalise at extraction time, and keep the currency in its own field. The same goes for quantities, weights and percentages.

Dates are ISO or they are ambiguous

04/09/2026 is the fourth of September in São Paulo and the ninth of April in New York. Force every date field to ISO 8601 (YYYY-MM-DD), and when a document only gives a relative term — “30 days” — model that as a separate integer field rather than jamming it into the date. Two clean fields, issue_date and payment_terms_days, let you compute the due date deterministically instead of asking the model to do arithmetic it may or may not do consistently.

Enums wherever the world is finite

Free-text classification fields are where silent drift lives. If a document type can be one of six things, declare the six values. The model then either picks one or fails visibly, which is far better than returning “fiscal service invoice (municipal)” on Tuesday and “NFS-e” on Wednesday and quietly breaking a routing rule that string-matches the first spelling. Always include an explicit escape value such as other or unknown so the model is never forced to squeeze a genuinely novel document into a wrong category.

Null, empty and zero are three different facts

A blank tax field can mean the document has no withholding, that the field was present but illegible, or that withholding was explicitly zero. Collapsing all three into an empty string destroys information the review queue needs. Give absence a vocabulary: nullable values plus a companion status, or a list of fields the extractor could not read. That single change is usually worth more than any prompt tuning, because it converts invisible failures into routable ones.

Units, currencies and scales belong in fields

Freight documents mix kilograms and tonnes; international invoices mix currencies; utilities mix kWh and MWh. Never encode the unit in the field name alone — put it in the payload next to the value. A weight plus weight_unit pair survives the day a carrier changes its template, while a field named weight_kg silently starts carrying tonnes.

Line items: the nesting decision

Repeating structures are where schemas either scale or collapse. Flattening line items into item_1_description, item_2_description and so on works until a document has more rows than you allowed, and it makes every downstream query awkward. Model repeated rows as an array of objects, give each row an explicit line number taken from the document, and resist the temptation to include totals per row that the model would have to compute.

Where arithmetic exists, extract the inputs and verify the output in code: if unit price times quantity does not match the printed line total, that is a finding worth raising, not something to paper over inside the extraction step. The same applies to the relationship between line totals and the invoice gross. A schema that keeps the components separate lets you build these consistency checks as ordinary assertions, which are cheap to run on every document and easy to explain to an auditor.

Give absence a vocabulary

The practical form of the null problem is small and worth copying directly. Each optional field gets a nullable value and a status drawn from a closed set — present, absent in the document, unreadable. Reviewers then see exactly what happened, and routing rules can treat “unreadable” as a human task while treating “absent” as a normal outcome.

Provenance is part of the schema

A value without a location is unverifiable. Carrying page number, bounding box and a confidence figure alongside each field turns review from a re-reading exercise into a glance: the reviewer sees the highlighted region and confirms or corrects in seconds. It also makes regressions diagnosable, because you can see whether a wrong value came from the wrong part of the page or from the right part read badly.

Keep provenance in a sibling object rather than wrapping every value in a record. Consumers that only want the data can ignore one key; consumers that need the audit trail have it in a predictable place. And treat confidence as a routing signal, not a truth claim — its job is to decide what a human looks at, not to be quoted as an accuracy number.

If two engineers reading the field name would fill it differently, the model will too.

— A working rule for extraction contracts

Versioning: the schema changes, the documents do not

Every schema is wrong in some small way by its third month — a field you did not anticipate, a type that needs widening, an enum missing a value. What matters is that the change is visible. Stamp schema_version into every payload, treat renames and type narrowings as breaking, and keep the old version readable for anything already stored. Without the stamp, you end up with a table holding three generations of the same field and no way to know which rows mean what.

Two rules keep this manageable. First, additive changes — a new optional field, a new enum value — are safe and get a minor bump; anything that changes the meaning of an existing key is a major bump and a new version string. Second, run every schema change against your evaluation set before it ships, because a change that improves one field often degrades a neighbouring one: splitting vendor into name and tax ID can make the name field noisier if the document prints them on one line. Measure, do not assume.

The version stamp only helps if the consuming code reads it. Put the check in the one place your system talks to the extraction service — the thin client wrapper described in the document extraction API integration guide — so an unexpected major version fails loudly at the boundary instead of three systems later.

The pre-flight checklist

• Every field name has exactly one reading — no engineer could fill it two ways.
• Monetary and quantity fields are numeric, with currency and unit in their own fields.
• All dates are ISO 8601; relative terms live in separate integer fields.
• Closed domains are enums, each with an explicit unknown/other escape value.
• Absence is modelled: nullable value plus a status from a closed set.
• Repeated rows are arrays of objects with a document-provided line number.
• No field requires arithmetic or a business rule the model has to infer.
• Provenance (page, bbox, confidence) sits in a sibling object.
• schema_version is stamped on every payload and bumped on every change.
• The schema was run against the eval set before it shipped.

Common questions

Should the schema be strict or permissive?

Strict on shape, permissive on content. Reject out-of-schema keys and wrong types outright — that is your first line of defence against malformed or manipulated output — but allow any field to come back null with a status. A schema that forbids nulls forces the model to invent plausible values, which is the worst possible failure mode because it is invisible.

How many fields is too many?

The constraint is rarely the model; it is the review burden. Every field you extract is a field someone may have to check, and fields nobody consumes are pure cost. Start from the columns that actually feed a decision downstream, and add rather than prune. If a field has not been read by any consumer after a month, it is a candidate for removal.

One schema per document type, or one universal schema?

One per document type, with a shared core. Invoices, freight bills and receipts have genuinely different fields, and a universal schema turns into a sparse object where most keys are null most of the time — which trains everyone to ignore nulls, exactly the habit you need to avoid. Share the common block (document type, dates, parties, totals, provenance, version) and specialise the rest.

Where do free-text fields fit?

Keep them, but keep them contained. A payment-terms note or an item description is real data and often the only place a crucial condition appears. Store it verbatim in a clearly named string field, never let it flow into a field that drives an action, and treat it as untrusted text on the way out — it is the natural carrier for anything hostile that arrives inside a document.

Where this leaves you

Schema design is the least glamorous part of a document pipeline and the highest-leverage. It is done before any model is chosen, it is testable with a text editor and a handful of real documents, and it determines the ceiling of everything downstream: what can be validated automatically, what a reviewer can confirm at a glance, what a consuming system can trust without defensive parsing. Spend the hour. Name the fields so that a careful reader and a model would fill them identically, give absence a vocabulary, keep the arithmetic in code, and stamp a version on the way out. The accuracy conversation gets much simpler once the contract stops being ambiguous.