Query transformation

Users ask vague, compound or conversational questions. "What about the enterprise plan?" means nothing to an index without the earlier turns. Transforming the query before retrieval often helps more than swapping embedders.

The big idea

The index stores documents. It cannot guess that "it" means last month's outage. You either rewrite the question into a standalone search string, or you search more than once.

REWRITE = """Given the chat, write a single search query that
finds documents to answer the latest user message.
Return only the query text.
Chat:
{chat}
"""

This is a prompt sketch, not a full client. The output is ordinary query text. Feed it to the same hybrid plus rerank path you already have.

HyDE (Hypothetical Document Embeddings) writes a plausible answer paragraph and embeds that paragraph instead of the raw question. It helps when questions are short and documents are long. The risk is a hallucinated hypothetical that retrieves the wrong neighborhood. Measure it.

Walking through a chat turn

Chat: "We use the Pro plan." then "What about the enterprise plan?"

A rewrite becomes enterprise plan pricing features SLA. Search now has nouns an index can match. Without the rewrite, the second turn is three words and a lot of pronouns.

For "Compare Acme and Beta on price and SLA," decomposition issues two queries, retrieves two small piles, and the generator writes the comparison. One blended query often misses one of the two vendors.

Common mistakes

• Turning on every transform at once. You cannot tell which one moved the metric, and you pay for all of them.
• Trusting HyDE without an eval. A fluent fake answer retrieves fluent fake neighbors.
• Rewriting when the question is already a good search string. You add latency for no gain.

Check yourself

1. The user says "what about the enterprise plan?" after talking about Pro. What should you search? A standalone query such as "enterprise plan pricing features SLA," not the raw three words.
2. Why can HyDE retrieve the wrong neighborhood? The hypothetical answer can be fluent and false, so its embedding points at the wrong documents.

Key takeaways

• Rewrite chat into a standalone query. Split comparisons. Expand ambiguous wording.
• HyDE embeds a drafted answer, not the question. Measure before you keep it.
• Each transform is an extra model call. Enable one at a time against a labeled set.

Further learning

When chunks themselves are orphaned from their document, fix ingestion with contextual retrieval.