Free lesson · AI Engineering

Choosing Embedding Models

Easy10 min readaillmrag

Choosing Embedding Models

The Default Choice That Cost Us 40% of Our Retrieval Quality

When my team at Meta first built an internal RAG system, we used OpenAI's text-embedding-ada-002 - the default at the time, and the model most tutorials recommended. It worked. Queries returned relevant documents. The RAG pipeline produced grounded answers. We shipped.

Three months later, a teammate benchmarked our retrieval against a newer model - text-embedding-3-small from OpenAI - and discovered that retrieval precision improved by 14% on our internal evaluation set. Not because ada-002 was bad, but because the newer model was better calibrated for the kind of text we were embedding - technical documentation with code snippets, API references, and domain-specific terminology. We swapped the model, re-embedded our corpus (a weekend job), and watched retrieval quality - and downstream RAG quality - jump measurably.

The lesson was not "always use the newest model." The lesson was that embedding model selection directly affects every downstream system that consumes the embeddings, and the difference between a well-chosen and a default-chosen model is larger than most engineers expect - often 10–20% on retrieval precision, which compounds through the RAG pipeline into answer quality differences that users notice.

This topic maps the embedding model landscape, gives you the selection framework, and teaches you how to benchmark models against your data rather than trusting generic leaderboards.

The Major Embedding Model Families

The embedding model landscape has matured rapidly. Three categories cover the options you will realistically evaluate.

Closed-Source API Models

Models available only through provider APIs. You send text, they return vectors. No access to weights, no self-hosting option.

OpenAI text-embedding-3 family. Two variants: text-embedding-3-small (1,536 dimensions, cheapest) and text-embedding-3-large (3,072 dimensions, strongest). Both support dimensionality reduction via the dimensions parameter - you can request 256, 512, or 1,024 dimensions from the large model, trading quality for storage and compute savings. OpenAI embeddings are the most widely deployed, the most tutorial-referenced, and a reasonable default for English-language applications.

Cohere Embed v3. Cohere's embedding model is competitive with OpenAI's and offers a notable feature: input type specification. You tell the API whether you are embedding a "search_document" or a "search_query," and it adjusts the embedding accordingly. This asymmetric embedding - different representations for documents versus queries - can improve retrieval precision for search use cases. Cohere also has strong multilingual support.

Google Gecko / Vertex AI embeddings. Competitive quality, integrated with Google Cloud infrastructure. Worth evaluating if your stack is already on GCP.

Voyage AI. A specialist embedding provider that has consistently ranked high on the MTEB benchmark. Their code-specific model (voyage-code-2) is particularly strong for codebases - relevant if your RAG system indexes code repositories.

Open-Source Models

Models you can download and run yourself. Weights are public; inference happens on your infrastructure.

BGE (BAAI General Embedding). One of the strongest open-source embedding families. The bge-large-en-v1.5 model is a commonly-cited baseline. Chinese-language variants are particularly strong. The BGE models use an instruction-prefix pattern: prepend "Represent this sentence:" to documents and "Represent this query:" to queries for best performance - a manual version of Cohere's input type feature.

E5 (Embeddings from Bidirectional Encoder Representations). Microsoft's open embedding family. e5-large-v2 and the newer multilingual variants are competitive with closed-source options. The instruction-prefix pattern applies here too: "query:" for queries, "passage:" for documents.

GTE (General Text Embedding from Alibaba). Strong all-around, particularly for multilingual applications. GTE models are available in multiple sizes from small (384 dimensions) to large (1,024 dimensions).

Nomic Embed. Open-source, competitive quality, with a focus on long-context embedding (8,192 token context length compared to the 512-token limit of many older models).

The MTEB Benchmark

The Massive Text Embedding Benchmark (MTEB) is the standard leaderboard for comparing embedding models across tasks: retrieval, classification, clustering, semantic similarity, and more. It is useful directionally - a model that ranks top-10 on MTEB is likely strong - but it has the same limitation as any benchmark: it measures generic capability, not performance on your specific data.

A model that ranks #3 on MTEB may rank #8 on your internal documentation corpus if your documents have domain-specific vocabulary the model was not trained on. The benchmark is a starting point for your shortlist, not a decision tool.

Warning: Do not select an embedding model solely based on MTEB rankings. Use MTEB to shortlist 3–4 candidates, then benchmark those candidates against your own data with your own evaluation metric. The 30 minutes spent running your own benchmark saves you from discovering a domain mismatch three months into production.

The Selection Framework

Four dimensions matter for embedding model selection. Evaluate them in this order because each constrains the next.

Dimension 1: Quality on Your Data

This is the only dimension that ultimately matters, and it is the one most engineers skip. Here is the benchmark protocol I use:

1. Collect 50–100 query-document pairs from your domain. Each pair: a query and its known-relevant document.
2. Embed all queries and all documents using each candidate model.
3. For each query, rank all documents by cosine similarity.
4. Measure retrieval precision: how often is the correct document in the top 1, top 3, top 5?
5. The model with the highest precision on your data is the right choice, subject to the other dimensions.

def benchmark_embedding_model(model, queries, documents, relevance_map):
    """Benchmark an embedding model on your domain data."""
    doc_embeddings = [model.embed(doc) for doc in documents]
    results = {"top1": 0, "top3": 0, "top5": 0}
    
    for query, relevant_doc_ids in relevance_map.items():
        query_embedding = model.embed(query)
        similarities = [
            (i, cosine_similarity(query_embedding, doc_emb))
            for i, doc_emb in enumerate(doc_embeddings)
        ]
        ranked = sorted(similarities, key=lambda x: -x[1])
        top_ids = [r[0] for r in ranked[:5]]
        
        if any(rid in top_ids[:1] for rid in relevant_doc_ids):
            results["top1"] += 1
        if any(rid in top_ids[:3] for rid in relevant_doc_ids):
            results["top3"] += 1
        if any(rid in top_ids[:5] for rid in relevant_doc_ids):
            results["top5"] += 1
    
    n = len(relevance_map)
    return {k: v / n for k, v in results.items()}

This takes 30 minutes to set up and saves months of "why is retrieval quality bad?" debugging later. Do it.

Dimension 2: Dimensionality

Embedding dimensions range from 256 to 3,072 across available models. Higher dimensions capture more nuance but cost more - in storage, in computation, and in vector database expenses.

The storage implications compound at scale: 1 million documents at 1,536 dimensions (float32) is roughly 6 GB of vectors. At 3,072 dimensions, it is 12 GB. The vector database must hold these in memory (or fast storage) for low-latency search. Dimensionality is a cost-quality trade-off, and the right point depends on your corpus size and your precision requirements.

OpenAI's text-embedding-3-large with the dimensions parameter lets you embed at 3,072 dimensions during benchmarking and deploy at 1,024 or 512 if the quality reduction is acceptable - measuring before committing.

Dimension 3: Multilingual Capability

If your application serves users or documents in multiple languages, the embedding model must handle them. Not all models do - and "handles multiple languages" has a spectrum:

Weak multilingual: embeds non-English text but produces lower-quality vectors compared to English. Retrieval precision drops 15–30% for languages not well-represented in training data.

Strong multilingual: produces comparable-quality vectors across languages, and crucially supports cross-lingual retrieval - a query in English can retrieve a document in Spanish, because the embedding space aligns meaning across languages. Cohere Embed v3, E5-multilingual, and GTE-multilingual all offer this.

Remember topic 2.2's multilingual tokenisation cost: non-English text tokenises into more tokens, so embedding cost per document is higher for non-English content. Factor this into your cost model.

Dimension 4: Cost

Embedding costs are per-token, like LLM costs, but dramatically cheaper - typically 10–50x cheaper than the cheapest chat model. The cost is paid twice: once to embed your document corpus (a one-time-ish cost, repeated when you re-embed after model changes) and once per query at request time.

graph TD
    A[Model Selection<br/>Framework] --> B[1. Quality on Your Data<br/>Benchmark 3-4 candidates<br/>on 50-100 domain pairs]
    B --> C[2. Dimensionality<br/>256-3072 dimensions<br/>trade storage vs quality]
    C --> D[3. Multilingual<br/>Cross-lingual retrieval?<br/>Language coverage?]
    D --> E[4. Cost<br/>Per-token embedding price<br/>× corpus size × query volume]
    E --> F[Decision:<br/>Best quality/cost on YOUR data]
    
    B -.->|"Do NOT skip this step"| B

This diagram shows the embedding model selection framework as a four-step sequence. Step 1, quality on your data, benchmarks 3 to 4 candidates on 50 to 100 domain-specific pairs. Step 2, dimensionality, trades storage cost against quality across the 256 to 3072 dimension range. Step 3, multilingual capability, evaluates cross-lingual retrieval and language coverage. Step 4, cost, computes per-token embedding price multiplied by corpus size and query volume. The decision at the bottom chooses the best quality-to-cost ratio on your specific data. A dotted self-loop on step 1 emphasises that this step should not be skipped.

At OpenAI's text-embedding-3-small pricing (~$0.02 per million tokens), embedding 100,000 documents at 500 tokens each costs

The exception: very large corpora (tens of millions of documents) where the one-time embedding cost and the ongoing storage cost become significant line items. At that scale, a smaller model (fewer dimensions, cheaper per-token) may be the right economic choice if it meets your quality threshold.

Context Length: The Embedding Model's Hidden Constraint

One specification that gets less attention than dimensions but affects your pipeline just as much: the embedding model's maximum context length - the longest text it can process in a single embedding call.

Older models (like ada-002) had a 8,191-token limit. Many popular open-source models (BGE, E5) default to 512 tokens - which means any text longer than roughly 380 words is silently truncated by the model, and the embedding only represents the beginning of the text. If your document chunks are 800 tokens and your embedding model's context is 512 tokens, the last 288 tokens of every chunk are invisible to the embedding. Your retrieval system cannot find content that the embedding model never saw.

Newer models have addressed this: OpenAI's text-embedding-3 family handles 8,191 tokens, Nomic Embed handles 8,192, and some models go further. But the constraint still interacts with your chunking strategy from topic 5.4: your chunk size must be at or below your embedding model's context length. If you choose a 512-token model, your chunks must be 512 tokens or fewer. If you choose an 8,192-token model, you have more flexibility - but remember from topic 5.1 that very large chunks produce blurry embeddings that hurt retrieval precision.

The practical check: before finalising your chunking strategy, confirm your embedding model's context length and ensure your largest chunks fit within it. This is a five-minute verification that prevents a class of retrieval bugs where "the answer is in the chunk but the search doesn't find it" - because the answer was in the truncated tail the model never embedded.

The Re-Embedding Decision

Changing your embedding model means re-embedding your entire document corpus. You cannot mix embeddings from different models in the same vector database - the vector spaces are incompatible, and cosine similarity between vectors from different models is meaningless.

Re-embedding is not difficult - it is a batch job that processes your corpus through the new model - but it has costs: compute time, API cost (if using a hosted model), and a brief period where your index is stale during the rebuild. For a 100,000-document corpus, re-embedding takes minutes to hours depending on the model and your parallelism. For a 10-million-document corpus, it takes hours to days and costs significantly more.

The implication: choose your embedding model carefully because switching has a real cost. The 30-minute benchmark from Dimension 1 is cheap insurance against a re-embedding you could have avoided.

Tip: Version your embeddings. When you embed your corpus, store the model name and version alongside the vectors. When you re-embed, create a new index rather than overwriting - so you can A/B test retrieval quality between the old and new embeddings before committing. The same version-and-evaluate discipline from topic 4.5 (prompt management) applies to embedding infrastructure.

sequenceDiagram
    participant D as Document Corpus
    participant E as Embedding Model v1
    participant V as Vector DB (Index v1)
    participant Q as Query Pipeline

    D->>E: Batch embed all documents
    E->>V: Store vectors + metadata
    Q->>E: Embed query at request time
    Q->>V: Search by cosine similarity
    V->>Q: Top-k results

    Note over E,V: Model upgrade: text-embedding-3-small → 3-large
    
    participant E2 as Embedding Model v2
    participant V2 as Vector DB (Index v2)
    
    D->>E2: Re-embed entire corpus
    E2->>V2: Store in new index
    Q->>V2: A/B test: query both indexes
    Note over V,V2: Compare retrieval quality before switching

This sequence diagram shows the embedding lifecycle and the re-embedding decision. Documents are batch-embedded with model v1 and stored in vector database index v1. Queries are embedded at request time and searched against the index. When upgrading to a new embedding model v2, the entire corpus must be re-embedded into a new index v2. Both indexes are queried in an A/B test to compare retrieval quality before committing to the switch.

Interview: The interview scenario: "You are building a RAG system for a legal document corpus. How do you choose your embedding model?" Strong candidates describe the framework: shortlist from MTEB, benchmark on domain-specific legal query-document pairs, evaluate multilingual if the corpus includes multiple jurisdictions, choose dimensionality based on corpus size and precision requirements, and note the re-embedding cost as a switching deterrent that makes the initial choice important. Candidates who say "I'd use OpenAI's default" without benchmarking miss the point: the default may be fine, but you should know it is fine from your own data, not from a tutorial.

Exercise: Benchmark Your Embedding Models

Exercise: Run a head-to-head benchmark on your own data:

1. Collect 20 query-document pairs from a domain you know (your codebase, your notes, your company wiki - anything with a natural "question and answer" structure).
2. Embed all queries and documents using two different models (e.g., OpenAI text-embedding-3-small and text-embedding-3-large, or an open-source model via Hugging Face).
3. Compute top-1 and top-3 retrieval precision for each model using the benchmark function from this topic.
4. Compare dimensions and compute the storage difference for a 100K-document corpus.
5. Challenge: compute the total cost of each model for your scenario - one-time corpus embedding + daily query embedding at your projected volume. Plot quality (precision) against total cost. Identify the Pareto-optimal choice.

What You Should Take From This Lesson

Embedding model selection directly affects every downstream component - retrieval quality, RAG accuracy, and ultimately user experience. Select by benchmarking 3–4 candidates on your own domain data, not by MTEB ranking alone. Evaluate across four dimensions: quality on your data (primary), dimensionality (storage-quality trade-off), multilingual capability (if needed), and cost (usually negligible, but significant at scale). Changing models requires re-embedding the entire corpus, so the initial choice carries switching costs that make benchmarking cheap insurance. Version your embeddings, A/B test before committing to a model change, and treat the embedding model as a first-class infrastructure decision.

Next: where those vectors live. Topic 5.3 covers vector databases - the storage and search infrastructure that makes embedding-based retrieval fast, scalable, and production-ready.

Back to AI Engineering