Hybrid search with reciprocal rank fusion

Keyword search misses meaning. A query like "how to stop my laptop overheating" shares no words with an article titled "Reducing CPU thermal throttling". Vector search matches meaning, but it is weaker on exact identifiers such as part numbers and rare names. Hybrid search runs both and merges the two ranked lists.

The big idea

Two judges rank the same contestants using different criteria. You cannot average their scores if one scores out of 10 and the other out of 100. But you can combine their rankings, because a rank means the same thing on any scale.

The picture breaks when one judge is poor. Fusion rewards documents that both judges like, so it works best when both lists are reasonable.

Walking through it

Reciprocal rank fusion (RRF) gives each document a score from each list it appears in:

RRF(doc) = sum over lists of  1 / (k + rank)      with k = 60 typically

A document missing from a list gets nothing from it.

def rrf(rank_lists, k=60):
    scores = {}
    for ranks in rank_lists:
        for doc, rank in ranks.items():
            scores[doc] = scores.get(doc, 0) + 1 / (k + rank)
    return sorted(scores.items(), key=lambda x: -x[1])

bm25 = {"X": 1, "Y": 3, "Z": 2}   # ranks in keyword results
vector = {"X": 5, "Y": 2}         # ranks in vector results; Z is absent
for doc, score in rrf([bm25, vector]):
    print(doc, f"{score:.5f}")

Output:

Y 0.03200
X 0.03178
Z 0.01613

The arithmetic, step by step:

X: 1/(60+1) + 1/(60+5) = 0.01639 + 0.01538 = 0.03178
Y: 1/(60+3) + 1/(60+2) = 0.01587 + 0.01613 = 0.03200
Z: 1/(60+2)            = 0.01613

Y edges out X. X ranks first in keyword search but fifth in vector search. Y is third and second, so it is strong in both lists. Z is strong in only one list, so it falls behind both.

Why it is practical

RRF needs only ranks, not comparable scores. You do not have to calibrate BM25 against cosine similarity. Elasticsearch, OpenSearch, Vespa and most vector databases support hybrid queries. In retrieval-augmented generation, the fused candidates are often reranked by a cross-encoder model before they reach the language model.

Common mistakes

• Averaging raw scores. BM25 and vector similarity live on different scales, so the average means little.
• Forgetting that missing means zero. A document absent from one list only gets credit from the other.
• Dropping keyword search for embeddings. Part numbers and rare names still need exact matching.
• Shipping without a test set. Judge the fused results on real queries before and after the change.

Check yourself

1. Why does Y beat X even though X ranks first in keyword search?

Answer: Y ranks third and second, so its two contributions sum to 0.03200, against 0.03178 for X, which ranks first and fifth.

2. Why use ranks instead of raw scores?

Answer: Keyword and vector scores are on different scales, but a rank position means the same thing in both lists.

Key takeaways

• Keyword search catches exact terms. Vector search catches meaning. Use both.
• RRF adds 1 / (k + rank) across lists, with k = 60 as a common default.
• Documents that rank well in both lists beat documents strong in only one.
• In retrieval-augmented generation, rerank the fused candidates before using them.

Further learning

Return to the search systems overview for how the pieces fit together.