Extraction schemas and validators

The model fills a versioned schema per document type. Constrained decoding cuts invalid JSON. Your validators still have to run. Schema-valid is not business-valid. sum(line_items) == total_cents is a code check, not a hope in the prompt.

The big idea

Keep validators in code, not in the prompt. Prompts approximate. Validators enforce. When a supplier uses tax-inclusive totals, encode that vendor rule in a table keyed by supplier_id, not in a vague prompt sentence.

Ask finance which fields are load-bearing before you tune prompts. Amounts, tax IDs, bank details, contract end dates. Write that list into the schema description and the review UI so everyone optimizes the same objective.

Ask the model for optional evidence: page number or quoted snippet per field. Store it for auditors and for the review UI (highlight the crop).

Walking through an invoice schema

{
  "type": "object",
  "required": ["supplier_name", "invoice_id", "invoice_date", "currency", "total_cents", "line_items"],
  "properties": {
    "invoice_date": { "type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$" },
    "currency": { "type": "string", "enum": ["USD", "EUR", "GBP", "INR"] },
    "total_cents": { "type": "integer" }
  }
}

def validate_invoice(doc: dict) -> list[str]:
    errors = []
    total = sum(item["amount_cents"] for item in doc["line_items"])
    if total != doc["total_cents"]:
        errors.append("line_items_sum_mismatch")
    if doc["total_cents"] < 0:
        errors.append("negative_total")
    return errors


sample = {
    "total_cents": 1000,
    "line_items": [{"amount_cents": 400}, {"amount_cents": 500}],
}
print(validate_invoice(sample))

Output:

['line_items_sum_mismatch']

400 + 500 is 900, not 1000. The JSON was valid. The invoice is not. That row goes to a human, not to the ERP.

Document inputs vary. Digital PDFs with a text layer: extract text, send text plus schema. Scanned pages: OCR first, or a native multimodal PDF/image API. Mixed packets: detect per page. Email plus attachments: classify each part and stitch with rules. Multimodal models read layout (amounts in the top-right) better than naive OCR dumps, and cost more. Always store the bytes you scored so reprocessing is reproducible.

Long contracts may exceed context. Per-section extraction with a merge step, map-reduce over chunks with an invoice_id continuity check, or extract only the clause types you need.

Common mistakes

• Trusting constrained JSON as a correct total.
• Vendor exceptions in the prompt instead of a supplier_id rule table.
• No evidence field. Reviewers cannot see why a value was predicted.

Check yourself

1. The model returns valid JSON and line items sum to 900 against total_cents 1000. Auto-commit? No. The business rule failed. Queue it.
2. Why store cents as integers? Decimals such as 100.00 versus 10000 are a classic swap. Integers plus a currency enum catch more of that.

Key takeaways

• Version the schema per document type.
• Constrained decoding, then business-rule validators in code.
• Ask for page or snippet evidence per field.
• Store the bytes you scored. Tier text versus multimodal by input type.

Further learning

Cost at a million documents a month is extraction tiering.