IVF, PQ and DiskANN

HNSW is the default until memory or corpus size says otherwise. Three older and complementary ideas extend the scale: cluster the space (IVF), compress each vector (PQ), or keep the graph on SSD (DiskANN).

Three approaches

IVF (inverted file) clusters vectors, often with k-means, into nlist centroids. At query time you probe the nprobe nearest clusters and scan only those lists. Less RAM than a dense HNSW graph at huge scale. Recall depends heavily on nprobe.

Product quantization (PQ) compresses each vector into a short code by splitting dimensions into subspaces and quantizing each. Distance is approximated between the query and the codes. Huge memory win — think 8–32 bytes per vector instead of 4×d — with some quality loss. IVF+PQ (IVFPQ) is a classic FAISS recipe for billion-scale search.

DiskANN and related disk-based graphs keep most of the index on SSD with carefully laid-out graph traversal, so the working set exceeds RAM. Use when the corpus does not fit in memory but you still need high recall — common past tens of millions of high-dim vectors on a single node.

Quantizing to float16 halves the raw HNSW footprint, but check whether your engine and recall targets allow it. PQ can go much lower with more quality risk.

Walking through a compression choice

10M vectors × 1,024 dims × 4 bytes ≈ 40 GB raw.

float32 HNSW     ~80–120 GiB RAM
float16 vectors  ~40 GB raw, plus graph
PQ 16 bytes/vec  10e6 × 16 = 160 MB of codes
                 (plus inverted lists / graph metadata)

PQ's 160 MB of codes is why billion-scale search exists on one box. The cost is recall. Measure it. If recall@10 drops from 0.96 to 0.80, the LLM will look worse and you will waste weeks tuning prompts.

A sensible rollout at work:

1. Flat index plus an eval harness while the corpus is small.
2. HNSW in Postgres or a single-node vector store once latency demands it.
3. Dedicated cluster / PQ / disk when finance asks why the vector box needs 256 GB RAM.

Common mistakes

• Jumping to PQ before you have measured HNSW recall. You will not know what you lost.
• Leaving nprobe at the default after the corpus grows. Recall cliffs look like "RAG got worse."
• Mixing metrics. Building with L2 and querying with cosine, or ignoring how the embedder was normalized.

Check yourself

1. Why does IVF scan only some vectors? It probes the nprobe nearest clusters and ignores the rest.
2. 10M vectors at 16 bytes each is how much for the codes alone? 160 MB. That is the PQ memory win versus ~40 GB of float32.

Key takeaways

• IVF clusters first, then scans a few lists. Tune nprobe.
• PQ compresses vectors to a few bytes. Measure the recall drop.
• DiskANN-style graphs keep the working set on SSD when RAM runs out.
• Move down this list only when HNSW memory or build time forces you.

Further learning

Pinecone Learning Center. Next, the query that every real product actually runs: filtered vector search.