Choosing an embedding model

There are many embedding models, and public leaderboards rank them on averages across many tasks. Your documents may rank the models differently. The practical answer is a small test on your own data, judged against a few constraints: quality, dimension, input length, language, cost and storage.

The menu, as of 2025 to 2026

Six questions to ask

1. Quality on your task. Start from the MTEB leaderboard. It averages many tasks, so confirm the ranking with a small test on your documents.
2. Dimension. Storage and index memory grow with dimension.
3. Maximum input length. Many models cap around 512 tokens. Longer chunks are cut off silently.
4. Languages. English-only models degrade on other languages. Pick a multilingual model if your corpus mixes them.
5. Cost and latency. API embeddings are cheap at small scale. Re-embedding tens of millions of documents often is not.
6. Truncation. Matryoshka models let you store shorter vectors and accept a small quality drop.

Walking through the storage math

Suppose you have 2,000,000 chunks and a 1,024-dimension model that stores each number as a 4-byte float.

Raw vectors:       2,000,000 × 1,024 × 4 bytes = 8.192 GB ≈ 7.6 GiB
Truncated to 256:  2,000,000 × 256 × 4 bytes   = 2.048 GB ≈ 1.9 GiB

Truncating from 1,024 to 256 numbers cuts raw storage by a factor of four. An in-memory HNSW index adds overhead on top. Plan RAM at roughly 2 to 4 times the raw vector size.

How to run a small bake-off

1. Collect 50 to 200 real queries, and mark which chunks should answer each one.
2. Embed the corpus with two candidate models. Start with two, not ten.
3. Measure recall@5 and MRR for each model.
4. Record embedding latency and cost per million tokens.
5. Pick the best trade-off for your constraints, and put the model ID in config.

Common mistakes

• Picking by leaderboard rank alone. The ranking may not match your documents.
• Sending chunks longer than the model's limit. The index then holds only the first part of each chunk.
• Mixing vectors from two models in one index.

Check yourself

1. Your chunks average 2,000 tokens and the model caps at 512. What does the index hold? Only about the first quarter of each chunk, since 512 is about 26% of 2,000. The rest is dropped without an error.
2. You truncate 1,024 dimensions to 256 to save storage. What must you measure? Retrieval quality on your own eval set, since truncation costs some accuracy.

Key takeaways

• Choose a model with a bake-off on your own queries and documents.
• Check the dimension, input limit, languages and cost, not just quality.
• Storage scales with the number of vectors times their dimension.
• Re-embedding is needed whenever you change models.

Further learning

Next, see how embeddings show up in retrieval on embeddings.