Extraction tiering and batch cost

At about 1 million documents a month, sync APIs are usually the wrong default. Peaks matter more than monthly averages. 1M docs/month is about 0.38 docs/s mean, but a partner may drop 50,000 files at 09:00 Monday. Size workers for peak. Object-storage ingest should acknowledge quickly, then queues absorb work.

The big idea

Prioritize payment-critical invoices ahead of archive backfills. Separate queues per tier so human review latency does not block cheap auto-commits. Track age of oldest unprocessed doc as an SLO, not only docs per minute.

Batch API jobs need their own scheduler: submit N files per job within provider limits, poll or use webhooks, write results idempotently.

Classify into tiers with layout cues (text extractable? known vendor?) before spending frontier tokens.

Walking through the bill

Assume 4,000 input tokens plus 400 output tokens per doc on a mid-tier model at roughly $3 / M input and $15 / M output (order of magnitude as of 2025 to 2026).

Cost/doc ≈ 4000/1e6 * 3 + 400/1e6 * 15 = $0.018
At 1M docs → about $18,000 / month
Batch discount around 50% → about $9,000 / month
Tier 70% to a cheap model at about 1/5 the price
  for easy invoices → blended well under $5,000 to $8,000
  before human review labor

If Tier C (frontier multimodal) share creeps up, invest in better classification or vendor templates rather than accepting a forever-high bill. A monthly cost review with finance keeps extraction from becoming an unexamined line item that grows with volume alone.

Dead-letter queues for poison PDFs (encrypted, corrupt, 0-byte) keep workers healthy. Alert when dead letters grow instead of retrying forever.

Common mistakes

• One frontier model for every PDF. Clean digital invoices do not need it.
• Sizing for the monthly average. Monday's 50,000-file drop is the real load.
• One shared queue for auto-commit and human review. Humans become the bottleneck for everything.

Check yourself

1. 4,000 input tokens at $3 / MTok and 400 output at $15 / MTok. Cost per doc? $0.012 + $0.006 = $0.018.
2. Why a separate queue per tier? Human review of hard scans must not stall cheap auto-commits.

Key takeaways

• Batch APIs plus model tiering make million-doc cost predictable.
• Size for peaks. SLO on age of oldest unprocessed doc.
• Classify before you spend frontier tokens.
• Dead-letter poison files. Do not retry them forever.

Further learning

Which docs a human still sees is extraction confidence.