What Is An Embedding, AI Engineer Stage 1

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings produce vectors that sit close together in a high-dimensional space — which is what makes it possible to search by meaning instead of by keyword. Embeddings are the mathematical foundation that RAG, semantic search, clustering, and recommendations are all built on.

Why this appears in interviews

Embeddings are upstream of every retrieval system. Interviewers ask about them to test whether you understand why retrieval works — and, more importantly, why it fails. Most "the RAG system returns the wrong documents" bugs trace back to an embedding decision: the wrong model, a mismatch between query and document embeddings, or an assumption that semantic similarity equals relevance. If you can't reason about embeddings, you can't debug retrieval.

The mental model

Imagine plotting every word in the language on a map, positioned so that words with similar meanings are neighbors. "Dog" sits near "puppy." "King" sits near "queen." "Paris" and "France" are closer to each other than either is to "banana."

A 2D scatter plot of word embeddings clustered by semantic similarity

Now scale that from words to whole sentences. "How do I reset my password?" and "I forgot my login credentials" land right next to each other — even though they share no words. That is the entire point: embeddings capture meaning, so a search can match "forgot my login" to a document titled "password reset" that a keyword search would miss.

The map isn't 2D, of course. A real embedding is a list of hundreds or thousands of numbers — 1536 for OpenAI's text-embedding-3-small. Each number is a coordinate along one dimension. You can't visualize 1536 dimensions, but the math of "how close are these two points?" works exactly the same as on a 2D map.

How embeddings are created

An embedding model — a different model from the one that generates text — takes text in and returns a fixed-length vector out:

def embed(text):
    resp = client.embeddings.create(
        model="text-embedding-3-small",
        input=text,
    )
    return resp.data[0].embedding   # a list of 1536 floats

embed("How do I reset my password?")
# -> [0.021, -0.453, 0.891, ... 1536 numbers]

Two facts that matter in practice:

The individual numbers are meaningless. No single dimension is "the password dimension." Meaning is encoded in the pattern across all dimensions, learned during training. You never interpret the numbers directly — you only compare vectors to each other.
The same model must embed both sides. If you embed your documents with text-embedding-3-small and your queries with text-embedding-3-large, the vectors live in different spaces and comparing them is meaningless. This single mistake silently destroys retrieval quality and is a favorite interview trap. Rule: one embedding model, used everywhere, and re-embed everything if you ever change it.

Measuring similarity — cosine similarity

To find "similar" text you compare vectors. The standard measure is cosine similarity, which measures the angle between two vectors:

1.0 — same direction (very similar meaning)
~0 — perpendicular (unrelated)
-1.0 — opposite direction

import numpy as np

def cosine_similarity(a, b):
    a, b = np.array(a), np.array(b)
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

Why cosine and not plain (Euclidean) distance? Because the magnitude of an embedding can vary for reasons unrelated to meaning (longer text, for example), while the direction carries the semantic signal. Cosine similarity ignores magnitude and compares only direction. In practice, providers often return normalized vectors, in which case cosine similarity and dot product are equivalent — but knowing why cosine is the default is the interview-level answer.

As a rough calibration for text-embedding-3 models: scores above ~0.8 usually mean highly relevant, and below ~0.5 usually mean unrelated — but these thresholds are model- and domain-specific and must be tuned, not memorized.

The crucial caveat: similar ≠ relevant

Embeddings capture semantic similarity, which is not the same as answering the question. "What is the capital of France?" and "What is the capital of Germany?" are extremely close in embedding space — same structure, same topic — but one does not answer the other. This gap is exactly why production retrieval adds reranking and hybrid search on top of raw embedding similarity (Stage 2). Naming this limitation is a strong signal in interviews.

Choosing an embedding model

| Model | Dimensions | Cost | Notes | |-------|-----------|------|-------| | OpenAI text-embedding-3-small | 1536 | low | Great default; ~90% of large's quality at a fraction of the cost | | OpenAI text-embedding-3-large | 3072 | higher | Best quality; heavier storage and slower search | | Cohere embed-v3 | 1024 | low | Strong quality; good multilingual support | | nomic-embed / open models | 768 | free (self-host) | Solid, keeps data in your VPC |

The decision axes: quality (retrieval accuracy on your data — always test), dimensions (more = more expressive but more storage and slower search), cost (you re-embed your whole corpus and every query), domain/multilingual fit, and hosted vs self-hosted (data residency). A useful trick: text-embedding-3 models support shortening dimensions (e.g. to 512) with a small quality hit to save storage and speed up search.

Common interview mistakes

Mistake 1: Using different models for documents and queries. The number-one silent retrieval killer. Same model everywhere.

Mistake 2: Confusing the embedding model with the generation model. They're separate systems with separate costs; embeddings never "generate" text.

Mistake 3: Assuming similarity means relevance. High cosine similarity can still be the wrong answer; production adds reranking and hybrid search.

Mistake 4: Ignoring dimensionality tradeoffs. Higher dimensions cost more storage and slow search for often-marginal quality gains — measure on your data.

Key vocabulary

Vector — An ordered list of numbers representing a point in high-dimensional space.
Embedding model — A model that converts text into a vector; separate from the generation model.
Cosine similarity — Similarity based on the angle between vectors (−1 to 1); ignores magnitude, compares direction.
Semantic search — Search by meaning rather than exact keyword match, powered by embedding similarity.
Embedding space — The shared high-dimensional space vectors live in; comparisons are only valid within one model's space.