Guide

Retrieval

Contextual retrieval

Situate every chunk with document and section context before you embed it, so "Revenue grew 3%" still names the company.

68 of 155 · 2 min read

Chunks lose document-level context when they are indexed alone. A chunk that says "Revenue grew 3% in Q3" is useless without knowing which company and which report. Contextual retrieval prepends a short, chunk-specific prefix before embedding and before BM25 indexing.

Concept

Before you start: chunking cuts documents into passages. Each passage becomes one vector. A prefix is extra text stored on the chunk so that vector still names its source.

The big idea

You would not file a photocopied sentence in a drawer without writing the report title on it. The same rule applies to chunks.

Original chunk:
Revenue grew 3% in Q3.

Contextualized chunk:
This chunk is from Acme Corp's 2024 Form 10-K, section MD&A, discussing
North America subscription revenue. Revenue grew 3% in Q3.

The mechanism is an ingestion upgrade. You re-embed after you add it. Pair it with hybrid BM25 for the full effect.

In Anthropic's reported evaluations (top-20-chunk retrieval failure rate, which is 1 − recall@20):

  • Contextual embeddings alone: about 35% fewer failures (for example 5.7% to 3.7%).
  • Contextual embeddings plus contextual BM25: about 49% fewer (to about 2.9%).
  • That hybrid plus reranking: about 67% fewer (to about 1.9%).

Treat these as published results on their corpora, not a guarantee on yours. The mechanism still holds: situate every chunk before you embed it. Prompt caching makes the offline pass cheaper when you use a frontier model to write the context sentences.

A lighter cousin needs no model. Prepend deterministic breadcrumbs: title, heading, product, version. Try that in a day. Escalate to model-written contexts when recall@k plateaus and error analysis shows orphaned pronouns ("it increased 3%") as a failure theme.

Common mistakes

  • Adding context only in the prompt, not in the index. Retrieval still matches the orphaned sentence.
  • Forgetting to re-embed. Old vectors were built from the bare chunks.
  • Skipping the breadcrumb prefix and jumping to an LLM writer. Title and heading often fix the same misses.

Check yourself

  1. Why is "Revenue grew 3% in Q3" a bad chunk to embed as-is? The vector has no company or report, so it can match any firm's Q3 notes.
  2. Is contextualization a query-time or ingest-time change? Ingest-time. You rewrite the chunk, then re-embed and re-index.

Key takeaways

  • A chunk must name its document and section, or retrieval cannot tell copies apart.
  • Start with title and heading prefixes. Add model-written context when evals stall on orphaned pronouns.
  • Re-embed after you add context. Combine with BM25 and a reranker.
  • Published failure-rate cuts are evidence for the mechanism, not a promise on your corpus.

Further learning

Introducing Contextual Retrieval (Anthropic)Explore how Anthropic enhances AI systems through advanced contextual retrieval methods. Learn about our approach to improving information access and relevance in large language models.anthropic.com

The map of when to use this lever sits on advanced RAG.

Advanced RAG

Hybrid search, reranking, query rewriting, contextual retrieval and measuring retrieval quality.

Chunking for RAG

How to cut documents into passages for retrieval, why naive character cuts break facts, and how size, overlap and context prefixes change what can be found.

The RAG ingestion pipeline

The offline half of RAG, from parsing documents to syncing edits, deletes and access rules, so the index never drifts from the source.