Extraction confidence and review

Models do not give well-calibrated probabilities out of the box. If "0.9 confidence" is only right 70% of the time, your auto-commit threshold is fiction. Fix the signal before you trust it.

The big idea

Practical confidence signals:

• Constrained decode success versus repair attempts
• Validator failures (arithmetic mismatch → low confidence)
• Second-model agreement on critical fields
• Retrieval or template match score for known vendors
• Self-reported confidence only if calibrated on your labeled set

Build a calibration plot on a held-out set: bucket predicted confidence versus actual field accuracy. Dual-model agreement on total_cents and invoice_id is often worth the extra batch cost compared to human minutes.

Route anything below threshold — or any critical-field disagreement — to humans. The review UI should show the document image, predicted fields, evidence highlights and one-click correct-and-approve. Every human correction becomes an eval label. Track mean time in review queue as an ops SLO so finance is not waiting on a silent backlog.

At 1M docs/month, even a 5% review rate is 50,000 human docs. Staffing must be planned. Reduce review volume with vendor templates and tier A models. Do not "just hire more" without measuring auto-commit error rates via audit sampling (for example 0.5% of auto-commits rechecked). If audit error rate exceeds budget, raise confidence thresholds immediately.

Walking through a disagreeing total

Model A reads total_cents = 109900. Model B reads 110000. The line-item sum is 109900. The executor treats the disagreement as low confidence even though both JSONs were valid. A reviewer sees the last-page crop, keeps 109900 and approves. That label goes back into the golden set.

If reviewers cannot see why a field was predicted, they will rubber-stamp errors at scale. Never hide uncertainty behind a green checkmark.

Common mistakes

• One global threshold for every field. Totals and descriptions are not the same risk.
• Self-reported confidence with no calibration plot.
• A review UI that hides the page crop. Reviewers guess, then approve.

Check yourself

1. "0.9 confidence" is right 70% of the time. What do you do? Stop auto-committing at 0.9. Fix the signal. Plot calibration.
2. Why audit 0.5% of auto-commits? To catch a silent rise in error rate before finance does.

Key takeaways

• Route on validators, agreement and calibrated signals, not vibes.
• Stricter thresholds on critical fields.
• Review UI shows image, fields, evidence and one-click correct.
• Sample auto-commits. Raise the bar when audit error exceeds budget.

Further learning

Re-running after a prompt fix without corrupting the ERP is extraction reprocessing.