A vector databaseVector databaseDatabase optimised for storing and searching embeddings by similarity using ANN algorithms.Learn more → is a database designed specifically to store and search embeddingsEmbeddingNumerical representation of text capturing semantic meaning. Similar texts produce similar vectors, enabling similarity search.Learn more → by similarity — it lets you ask "what stored vectors are most similar to this query vector?" efficiently at scale.
Why this appears in interviews
Every RAGRAGRetrieval-Augmented Generation — gives LLMs access to external knowledge by retrieving relevant documents before generating a response.Learn more → system needs somewhere to store and search embeddingsEmbeddingNumerical representation of text capturing semantic meaning. Similar texts produce similar vectors, enabling similarity search.Learn more →. Interviewers test vector databaseVector databaseDatabase optimised for storing and searching embeddings by similarity using ANN algorithms.Learn more → knowledge to see whether you understand why you cannot just use PostgreSQL for this problem (you can, but it does not scale), what the tradeoffs between different solutions are, and how approximate nearest neighbor search works.
The mental model
Imagine a library with millions of books, and you want to find the books most similar in topic to a book you are holding. A regular database lets you search by title, author, or keywords — exact matches only. A vector databaseVector databaseDatabase optimised for storing and searching embeddings by similarity using ANN algorithms.Learn more → lets you say "find me the books whose topic-vectors are closest to this book's topic-vector" — semantic similarity search.
The hard part: if you have 10 million vectors, comparing your query vector to all 10 million is too slow for real-time use. Vector databases solve this with Approximate Nearest Neighbor (ANN) algorithms — they trade a small amount of accuracy for dramatically faster search.
How HNSW works
HNSW (Hierarchical Navigable Small World) is the most widely used ANN algorithm. It works like navigating a map:
- At the top level, a sparse graph connects distant points ("cities")
- Each level below gets more detailed ("neighborhoods," "streets," "houses")
- To find nearest neighbors to a query, start at the top level, navigate toward the query, then zoom in at each level
- Reach approximate nearest neighbors without comparing against every point
This allows searching 10 million vectors in milliseconds instead of seconds.
Choosing between solutions
- Pinecone — Managed cloud service. Easy to start, no infrastructure. Gets expensive at scale. Good for startups and prototypes.
- Weaviate — Open source, can self-host. Supports hybrid search (vector + keyword) natively. Good for production systems.
- Qdrant — Open source, excellent performance per dollar. Rust-based, very fast. Good for teams that want control.
- pgvector — PostgreSQL extension. Not a dedicated vector databaseVector databaseDatabase optimised for storing and searching embeddings by similarity using ANN algorithms.Learn more → — vector search added to a relational database. Appropriate under ~1M vectors.
- Chroma — Easy setup, popular for prototypes and local development. Not production-ready for large scale.
Metadata filtering
A real-world query is rarely "find similar vectors across all my data." More often it is "find similar vectors from documents written after January 2024" or "from this specific customer." Vector databases support metadata filters — you attach metadata to each vector and filter on it alongside similarity search. Pre-filtering (filter first, then search) is more accurate and supported by most modern vector databases.
Common interview mistakes
Mistake 1: Defaulting to pgvector. pgvector is fine for small-scale applications but should not be your default answer for a system design question. Know when it is appropriate.
Mistake 2: Not knowing what ANN trades off. ANN algorithms are approximate — they might miss the truly closest vector occasionally. The accuracy/speed tradeoff is controlled by parameters like ef_construction in HNSW.
Mistake 3: Forgetting hybrid search. Pure vector search misses exact keyword matches. Production systems often need hybrid search (vector similarity + BM25) merged via reciprocal rank fusion.
Key vocabulary
- ANN (Approximate Nearest Neighbor) — Search algorithm that finds approximately nearest vectors, trading small accuracy loss for massive speed gains.
- HNSW — Hierarchical Navigable Small World, the most widely used ANN algorithm.
- Hybrid search — Combining vector similarity search with traditional keyword search (BM25).
- Index — The data structure built by a vector databaseVector databaseDatabase optimised for storing and searching embeddings by similarity using ANN algorithms.Learn more → to enable fast ANN search.