Vector Databases
Background
If you've been paying attention to anything in tech over the past few years, you've noticed "embeddings" everywhere. Search engines that actually understand what you mean. Recommendation systems that surface eerily relevant content. Chatbots that can retrieve information from massive document collections. All of these rely on the same underlying primitive: finding things that are similar to other things, fast.
This isn't actually a new concept. Vector databases and their related techniques have been around for a long time in recommendation systems. But the power of vector databases has been amplified by the rise of new machine learning techniques, unlocking a cool new set of infra applications.
Traditional databases are great at exact lookups — give me the user with ID 12345, find all orders placed on January 1st. But ask a traditional database "find me documents similar to this one" and you're in trouble. That's where vector databases come in.
This deep dive covers what vector databases are, how they work under the hood, and — most importantly — how to use them effectively in a system design interview. We'll go deep on the indexing algorithms that make similarity search fast, but we'll also be practical about when you actually need a dedicated vector database versus when a simple extension to your existing database will do the job.
Most system design interviews won't cover vector databases in this much depth. Those that do often don't care that you know the internals as much as they care about you knowing how and where to use them. If the algorithm detail below is overwhelming, skip to the Applications section and work backwards.
What's a Vector Anyway?
Before we talk about databases that store vectors, we need to understand what we're actually storing.
A vector (or embedding) is just an array of numbers that represents something. That "something" could be a word, a sentence, an image, a user, a product, or really anything you can feed into a machine learning model. The magic is that similar things end up with similar vectors.
// Two sentences that mean similar things
"The cat sat on the mat" → [0.12, -0.34, 0.78, ..., 0.45] // 1536 numbers
"A feline rested on a rug" → [0.11, -0.32, 0.79, ..., 0.44] // very similar!
// A sentence with different meaning
"The stock market crashed" → [-0.89, 0.12, -0.45, ..., 0.23] // very different
The typical embedding has somewhere between 128 and 1536 dimensions (OpenAI's text-embedding-3-large uses 3072). Each dimension captures some aspect of the meaning, though the individual dimensions aren't usually interpretable by humans. What matters is that the geometric relationships between vectors reflect semantic relationships between the things they represent.
Note we're being a bit hand-wavey about "similarity" here. Does similar mean the same color? Excerpts from the same book? A similar concept? That actually depends on the embedding model you're using.
Many applications use a pre-trained embedding model. For text, this might be OpenAI's embedding API, Sentence Transformers, or BERT. For images, models like CLIP or ResNet. These models are trained on diverse tasks such that the notion of similarity you care about is probably captured — think of it as a very vague "semantic" similarity. To you, the embedding model is an expensive GPU function: data goes in, a fixed-length vector comes out.
But "similarity" can be much more precise for the application. A very common pattern in recommendation systems is finding items likely to be purchased together. Diapers and bottles are only vaguely similar in that they're both baby-related, but profoundly similar in that new parents buy both. In these cases, a custom ML model creates embeddings specifically targeting this notion of "similarity."
Similarity Metrics
Once you have vectors, you need a way to measure how similar two vectors are.
| Metric | How it works | When to use |
|---|---|---|
| Euclidean distance (L2) | Straight-line distance between two points (Pythagorean theorem extended to N dimensions). Smaller = more similar. Cares about direction and magnitude. | Un-normalized embeddings, spatial data |
| Cosine similarity | Measures the angle between two vectors, ignoring magnitude. Same direction = 1, perpendicular = 0, opposite = -1. | Normalized embeddings (most text/image models) — the default choice |
| Dot product | Like cosine similarity but doesn't normalize for magnitude — cosine similarity × vector lengths. | Slightly faster to compute; used when magnitude carries signal |
| Hamming distance | Counts positions where two binary vectors differ (XOR + count). Extremely fast. | Binary embeddings, Locality Sensitive Hashing |
For an infra-style system design interview, the choice of similarity metric usually doesn't come up much — "we'd use cosine similarity or some appropriate similarity metric" puts you ahead of most candidates. In an ML system design interview, the choice matters more depending on your embedding model and use case. If your embedding model was trained on a specific objective (e.g., minimizing dot product distance between items bought together), that's your distance metric.
The Nearest Neighbor Problem
Once we have vectors and a similarity metric, we can start asking questions — usually framed as a query vector (think: your search term). For most applications, the query vector is the embedding of the item you're trying to find similar things to.
K-Nearest Neighbors (KNN)
Given a query vector, return the K most similar vectors in the collection.
The naive approach (exact KNN): compare your query against every vector, compute similarity scores, sort, return the top K. This is effectively O(n) where n is the number of vectors (k is small enough to ignore).
# Python Pseudocode
def exact_knn(query_vector, all_vectors, k):
heap = [] # min-heap by similarity
for vector in all_vectors:
similarity = compute_similarity(query_vector, vector)
if len(heap) < k:
heapq.heappush(heap, (similarity, vector.id))
elif similarity > heap[0][0]:
heapq.heapreplace(heap, (similarity, vector.id))
return sorted(heap, reverse=True)
For a million vectors with 1536 dimensions, that's about 6 billion floating point operations per query. This gets expensive and is too slow for many applications.
This pseudocode can be optimized: on a CPU, SIMD instructions compute similarity scores for 2–8 vectors at once. On a GPU, vectorized operations compute similarity for thousands of vectors at once. So if exact KNN is required, there are ways to make it faster. But most applications can tolerate a bit of inaccuracy to gain a lot of speed.
Approximate Nearest Neighbor (ANN)
What if we don't need the exact nearest neighbors — just vectors that are probably the nearest neighbors most of the time? This is ANN search, and it's the foundation of every practical vector database. We trade off accuracy for speed.
The key quality metric is recall: of the true top-K nearest neighbors, what fraction did we actually find? A recall of 0.95 means we found 95% of the true nearest neighbors — plenty good enough for most applications.
This forms the underpinning of all vector databases. They let you make a smooth tradeoff between three things:
- Recall — how accurate are our results?
- Latency — how fast can we return results?
- Memory — how much space does our index consume (especially RAM)?
How Vector Databases Work
So we've got vectors, similarity metrics, and we know brute-force search doesn't scale. How do vector databases actually make this fast? Clever data structures that let us skip most of the comparisons.
Indexing Strategies
The key to making a vector database work is computing indexes on the stored vectors. These indexes make retrieval faster and usually require tradeoffs — often compromising recall for big reductions in latency. They also introduce complexity for inserts, updates, and deletions.
There are many indexing strategies for ANN search. The tradeoffs between them are hard to reason about without running experiments on your data — it's rarely the case you can look at an application and prove one strategy is superior. This is actually one of the main benefits of using a vector database: it lets you swap between indexing strategies and tune parameters without rewriting your application. You'd typically maintain an evaluation set (queries with known good results) and measure recall/latency tradeoffs to find what works.
Most interviews won't approach the level of detail below. The important thing isn't that you can implement these algorithms — it's having an intuition about how they work.
HNSW (Hierarchical Navigable Small World)
HNSW is the most popular algorithm in production vector databases. If you remember one indexing strategy, make it this one.
The intuition is similar to skip lists. In a regular linked list, finding an element requires scanning every node — O(n). Skip lists solve this by adding "express lane" layers above the base list. The bottom layer has all elements; higher layers skip over most elements, keeping only a random subset. To search, you start at the top express lane, zoom forward until you'd overshoot, then drop a level and continue. This gets you O(log n) search in a linked list.
HNSW applies the same idea to graph-based nearest neighbor search. It builds a multi-layer graph where each node is a vector.
Think of it this way: when you insert a vector into an HNSW index, it becomes a node. The index finds the vectors most similar to your new vector and creates edges connecting them. So if you insert an embedding for "Taylor Swift", it gets connected to nearby embeddings like "Beyoncé" and "Ed Sheeran" — not because anyone manually linked them, but because their vectors are geometrically close in the embedding space.
The result is a graph where you can "walk" from any vector to similar vectors by following edges. Crucially, if two vectors are similar, there's likely a short path between them through the graph.
The same skip-list idea appears here. The bottom layer (Layer 0) contains all vectors, each connected to their nearest neighbors. But searching this dense graph is still slow for large datasets. So HNSW adds "express lane" layers on top. Each vector has some probability of being "promoted" to higher layers. The result is a hierarchy: Layer 0 has millions of nodes densely connected to their neighbors, Layer 1 might have tens of thousands, Layer 2 might have hundreds, and so on. Vectors in higher layers act as "long-range" connections letting you jump across the space quickly.
Searching HNSW:
1. Start at the top layer with a random entry point
2. Greedy search: move to whichever neighbor is closest to your query vector
3. Repeat until you can't get any closer at this layer
4. Drop down to the next layer (which has more nodes) and continue greedy search
5. At Layer 0, do a more thorough local search to find the K nearest neighbors
The top layers let you quickly "zoom in" to the right region of the space. By the time you reach Layer 0, you're already in the right neighborhood and only need to explore locally. This gives O(log n) search complexity with excellent recall — HNSW consistently achieves 95%+ recall with low latency, which is why it's become the default choice.
This isn't free. HNSW indexes are memory-hungry — you store the graph structure (all those edges) on top of the vectors themselves, roughly 2x the memory of raw vectors. Building the index is slow since you're constructing this elaborate graph structure. Inserts are relatively expensive because each new vector needs to find its place in the graph and establish connections at each layer.
Interview framing: "It builds a multi-layer graph where vectors are nodes. Search starts at a sparse top layer and greedily navigates toward the target, dropping to denser layers as it gets closer. It's like skip lists but for high-dimensional space."
IVF (Inverted File Index)
IVF takes a different approach. Instead of building a graph, it partitions your vectors into clusters using k-means clustering. Each cluster has a centroid (the center point), and vectors are assigned to their nearest centroid.
At query time, you first find the closest centroids to your query vector, then only search within those clusters. If you have 1000 clusters and search 10 of them, you've eliminated 99% of comparisons.
Why can't you just search the "right" cluster? Like geospatial indexing, the problem is the edges. If your search query is near the edge of a cluster, you need adjacent clusters too, or you'll miss results. In 2D space there's plenty of room to be "on an edge." In 1536D space, there are a lot more edges.
The parameter nprobe controls how many clusters you search. Higher nprobe means better recall but slower queries — a nice knob to tune the recall/latency tradeoff.
Probes are easily parallelized, so a single query doesn't necessarily take a latency hit as nprobe increases. But under load with enough concurrent requests to saturate compute, you'll see a latency hit for the average request.
IVF is faster to build than HNSW and handles inserts more gracefully (just assign the new vector to a cluster). It uses less memory since you're only storing cluster assignments, not a full graph. The downside is typically lower recall for the same latency, especially if your data isn't naturally clustered.
Locality Sensitive Hashing (LSH)
LSH takes a fundamentally different approach: instead of building a graph or clustering, it uses hash functions designed so that similar vectors are likely to hash to the same bucket. Regular hash functions try to avoid collisions — LSH hash functions are designed to cause collisions for similar items.
The most common approach for cosine similarity uses random hyperplanes (a plane in high-dimensional space). Imagine drawing a random line through your vector space. Every vector is either "above" or "below" that line — that's one bit of your hash. Do this with, say, 8 random hyperplanes and you get an 8-bit hash. Vectors close together will likely be on the same side of most hyperplanes, so they'll have similar (or identical) hashes.
We make this more robust with multiple hash tables using different random hyperplanes. A single table might miss similar vectors that fall on opposite sides of one hyperplane, but with multiple tables, the probability that similar vectors share at least one bucket increases dramatically.
At query time: hash your query vector in all tables, collect all candidates from matching buckets, then compute exact distances only on this (hopefully small) candidate set. More tables and more bits means better recall but more memory and slower queries.
LSH was popular before HNSW became dominant. It's simple to implement, handles high dimensions well, and has nice theoretical guarantees. In practice, HNSW usually achieves better recall for the same latency. LSH is still useful when you need:
- Very fast index building (just compute hashes)
- Streaming data where vectors arrive continuously
- Hamming distance similarity (LSH is natural here)
Annoy (Approximate Nearest Neighbors Oh Yeah)
Like LSH, the idea of cutting up the vector space with random hyperplanes extends to tree-based structures. Annoy (from Spotify) builds a forest of random projection trees. The idea is beautifully simple: recursively split your vector space with random hyperplanes until each leaf node contains a small number of vectors.
To build a tree: pick two random vectors, draw a hyperplane equidistant between them. All vectors on one side go into the left subtree, the other side into the right subtree. Repeat recursively until each leaf has few enough vectors (say, 100). The result is a binary tree where nearby vectors tend to end up in the same leaf or nearby leaves.
To search: traverse the tree toward the leaf matching your query vector. A single tree can make mistakes — your true nearest neighbor might have ended up on the other side of an early split. So Annoy builds a forest of many trees (typically 10–100), each with different random splits. At query time, search all trees, collect candidate leaves, compute exact distances on the union of candidates.
If you're familiar with classical ML, there's a strong analog to Random Forest here.
The killer feature of Annoy is memory mapping. The entire index is stored as a single file that can be mmap'd into memory — a low-level system call mapping a file into memory, very efficient compared to orchestrating file reads/writes from userspace. This means:
- Multiple processes can share the same index without copying
- You can work with indexes larger than RAM (the OS pages in what you need)
- Index loading is instant (just
mmap, no deserialization)
The downside: Annoy indexes are immutable. Once built, you can't add or remove vectors — you have to rebuild the entire index. Great for static datasets (like Spotify's music catalog that updates in batches), unsuitable for real-time applications with continuously arriving vectors.
Annoy was the go-to solution at many companies before HNSW took over. You'll still see it in production systems where the dataset is static and memory mapping is valuable.
Filtering and Hybrid Search
Now you're an expert on indexing vectors. Summary: vector search retrieves "similar" items, made efficient (though slightly inaccurate) via HNSW, IVF, and LSH. But what about retrieving items that match a specific query — not just similarity?
Real applications rarely want "find the 10 most similar items" without constraints. You usually want "find the 10 most similar items that are in stock" or "in the user's price range" or "published this year." This is filtered vector search, and it's trickier than it sounds. Two options:
- Post-filtering: Find the top-N similar vectors (N >> K), then filter down to K results. Problem: if your filter is restrictive, you might not find K results. You can increase N, but then you're doing more work.
- Pre-filtering: Filter first, then search only within the filtered set. Problem: you might not be able to use your fancy index structures on an arbitrary subset of data.
Which is best depends entirely on the data — it's safe to tell your interviewer "this is going to depend heavily on the data; I'd set up a benchmark with real data and see which works best." That said, when you can make coarse statements (e.g., vector search over a tiny subset of your data), pre-filtering is probably better, and interviewers like finding edge cases to test your intuition. Same class of problem as something like FB Post Search, with a multi-dimensional twist.
Most vector databases try to take this heavy lifting off your shoulders — some maintain multiple indexes for common filter combinations, others integrate filtering directly into index traversal.
How Three Popular Systems Handle It
Postgres's pgvector relies on Postgres's query planner. You write a normal SQL query with both a WHERE clause and an ORDER BY using vector distance. The planner decides whether to use the vector index, a B-tree index on your filter column, or some combination. For highly selective filters, it often skips the vector index entirely and does brute-force similarity on the filtered rows — actually faster in that case. The catch: pgvector doesn't do true "filtered HNSW traversal" — it's either/or, so you can get suboptimal plans when filter selectivity is in an awkward middle ground.
Elasticsearch has tighter integration. Its kNN search supports a filter parameter applied during index traversal, not after. Under the hood it combines HNSW with filtered candidate generation: ES first identifies candidate vectors from the HNSW graph, applies your filter, then continues exploring until it has enough filtered results. Highly restrictive filters slow down the search (more graph traversal needed), but you're guaranteed K results if they exist. ES also supports hybrid search natively — combine BM25 keyword scoring with vector similarity using sub_searches or rescore.
Purpose-built vector databases like Pinecone treat metadata filtering as a first-class feature. Every vector can have arbitrary metadata (up to 40KB), and filters are applied during the ANN search — not before or after. Pinecone builds specialized index structures — inverted indexes on metadata fields alongside the vector index — and intersects the metadata filter with the vector search in a single operation. Pinecone also lets you tune a "filter effort" parameter to trade off filter precision and latency.
Hybrid Search (Keyword + Vector)
There's a class of searches wanting both full-text and vector search simultaneously, often called hybrid search. A query like "red running shoes" might use keyword matching for "red" and "running shoes" while using vector similarity to find semantically related products. This often gives better results than either approach alone — accomplished by running both searches in parallel and merging results, or using clever merging strategies.
Inserts, Updates, and Index Maintenance
Vector databases are generally optimized for read-heavy workloads. Writes are more complicated, especially with sophisticated indexes like HNSW.
Inserts can work in real-time, but it's compute-expensive. Adding a new vector to an HNSW graph means finding its place and updating connections. With many inserts, the graph structure can degrade over time (new vectors might not be as well-connected as vectors present during the initial build). Similarly, with IVF, over time you may need to rebuild the index as clusters move and evolve, or risk performance degradation.
Many systems handle this with a hot/cold index pattern: a small "hot" index for recent inserts and a larger "cold" index for older data. Queries search both and merge results. Periodically, the hot index gets merged into the cold index with a full rebuild. The "hot" index need not be a full index at all — it can be a dumb list of unindexed entries that you exhaustively search.
Hot Index (recent inserts, exhaustive search) + Cold Index (bulk, HNSW/IVF)
↓ ↓
Query both, merge results
↓
Periodic merge: hot → cold (full rebuild)
This pattern is useful beyond vector databases. If your system needs to handle deletions that rarely happen, maintaining a small index of deleted items to check before returning results gives you time to clean up caches and precomputations without blocking reads.
Updates are usually implemented as delete + insert. Most systems use soft deletes (marking vectors as deleted) rather than actually removing them — deleted vectors still consume space and slow down queries until you rebuild or compact the index.
Index rebuilds can be slow. Building an HNSW index over millions of vectors can take hours. Strategies:
- Rolling rebuilds — build a new index alongside the old one, then swap
- Partitioned indexes — rebuild one partition at a time
- Background reindexing — doesn't block queries
If your embeddings change frequently (updating your ML model, or underlying data changes rapidly), think carefully about your update strategy. Batch updates with periodic rebuilds are often more practical than real-time updates.
This is one major difference between vector databases and traditional databases — traditional databases handle real-time updates because they're designed for heavy write loads; vector databases often are not. In interviews, this means discussing a deliberate rebuild strategy, including a "hot" side index.
Vector Database Options
There are a lot of choices, and the field moves quickly — solutions popular a few years ago are starting to show their age.
Practical advice: start simple. Counter-intuitively, you probably don't need a purpose-built vector database. Extensions to databases you're already using will handle millions of vectors just fine, avoiding the operational overhead of another system. Only reach for a dedicated vector DB when scale or features demand it.
Vector Extensions for Traditional DBs and Stores (Start Here)
| Option | When to use |
|---|---|
| pgvector | Already on PostgreSQL. Supports HNSW and IVF, handles millions of vectors. Real advantage: everything else you get for free — ACID transactions, familiar tooling, joining vector results with relational data ("find similar products that are also in stock and in the user's region" — one query). |
| Elasticsearch kNN | Already have Elasticsearch for search. Straightforward to add, excellent hybrid search (keyword + vector) out of the box. Downside: ES is already operationally complex, but incremental if you're running it anyway. |
| Redis Vector Search | Real-time, low-latency requirements. Redis is already common in most architectures; the vector extension is simple. Index options are simpler than dedicated vector DBs — fine for many use cases. |
| S3 Vector | New AWS offering — store vectors in S3, query via the S3 API. Good if already using S3 and want to avoid the complexity of a dedicated vector database. |
Purpose-Built Vector DBs (When You Need Scale)
Rule of thumb: if you're dealing with more than 100 million vectors, consider a purpose-built vector database.
| Option | Profile |
|---|---|
| Pinecone | Fully managed, serverless. No infrastructure to run — just call an API. Easiest to operate. Tradeoff: cost and less control. |
| Weaviate | Open source, good hybrid search support, GraphQL API. Reasonable middle ground between DIY and fully managed. |
| Milvus | Open source, built for serious scale — billions of vectors. Tradeoff: operational complexity, running a distributed system with multiple node types. |
| Qdrant | Open source, written in Rust, particularly good filtering support. Worth it if complex filtered queries are central to your use case. |
| Chroma | Lightweight, great for prototyping. Increasingly popular in the LLM/RAG space for how easy it is to get started. |
Using Vector Databases in Your Interview
Common Interview Scenarios
Vector databases show up almost exclusively adjacent to AI/ML system design questions. You'll usually know going in whether you're interviewing for an AI/ML-adjacent team. Patterns that effectively mandate a vector database:
- Semantic search — "Design a document search system" or "Design a code search tool." Natural language queries → relevant documents. Classic embedding + vector search.
- Recommendations — "Design a product recommendation system" or "Design a content recommendation feed." Find items similar to what the user engaged with, often combined with collaborative filtering.
- Image/video similarity — "Design reverse image search" or "Design a similar videos feature." Embed the media, search for similar embeddings.
- RAG systems — "Design a knowledge base Q&A system" or anything involving LLMs with custom data. Vector search retrieves relevant documents, the LLM synthesizes an answer.
- Deduplication — "Design a near-duplicate detection system." Plagiarism detection, similar support tickets, duplicate listings. Embed items, find items within a similarity threshold.
- Anomaly detection — "Design a fraud detection system." Embed transactions, find transactions dissimilar to normal patterns.
If you don't have an ML background, reading high-level ML System Design breakdowns for these systems builds intuition about where vector databases fit.
Architecture Patterns
Pattern 1: Vector DB as a separate service. Most common. Your application generates/retrieves an embedding, sends it to the vector service, gets back IDs of similar items, then fetches full item details from your primary database. Clean separation of concerns.
Pattern 2: Hybrid search. Query goes to both a keyword index (like Elasticsearch) and a vector index. Results merged with a ranking function. Good for search applications where both exact matches and semantic similarity matter.
Pattern 3: Two-stage retrieval. Vector search returns a large candidate set (maybe top 1000), then a more sophisticated (but slower) model reranks them for the final results. Common in recommendation systems where the reranker uses features the embedding doesn't capture.
Query → Embed → Vector Search (top 1000 candidates)
↓
Reranking Model (features: recency, popularity,
user history, business rules)
↓
Final top-K results
Stick with Pattern 1 for most problems. Unless the question specifically focuses on retrieval/ranking, you're more likely to rabbit-hole into new and challenging problems than to impress the interviewer by adding unnecessary complexity.
Key Design Decisions to Discuss
- Consistency requirements. Vector search results are usually okay to be slightly stale. New items might not be searchable for seconds or minutes. Mention this explicitly: "Vector search can be eventually consistent; we don't need the embedding immediately searchable after insert."
- Update strategy. How do embeddings get into the system? Real-time as items are created? Batch job running hourly? Depends on latency requirements.
- Filtering strategy. If your query involves filters, how do you handle them — pre-filter, post-filter, or hybrid? Especially important if filters are selective.
Far less common for an infra-style interview, but worth knowing:
- Embedding model selection. What model produces your embeddings? Affects dimension size, quality, latency. For text: "we'd use a sentence transformer model" or "OpenAI's embedding API." Don't go deeper unless it's an ML system design interview.
- Index type. "We'd use HNSW for best query performance" is usually the right answer unless you have specific constraints (very write-heavy, extremely large scale, memory constraints).
- Embedding updates. Occasionally you'll change or refresh the embedding model — this means a massive rebuild requiring careful orchestration. Ensure your APIs track which model produced each embedding, so you're never searching with the wrong embeddings.
Numbers to Know
| Metric | Value |
|---|---|
| Embedding dimensions | 128–1536 typical. OpenAI uses 1536; many open-source models use 384 or 768 |
| Memory per vector | 4 bytes/dimension for float32. A 1536-dim vector ≈ 6KB raw (bigger than you might think!) |
| 1M vectors @ 1536 dims | ~6GB just for vectors, more with index overhead (HNSW roughly doubles it) |
| Query latency | Sub-10ms achievable for well-tuned systems; 1–5ms common |
| Recall targets | 95%+ usually acceptable; 99%+ achievable but costs more in latency or memory |
| Throughput | Tens of thousands of QPS per node realistic for in-memory indexes |
Gotchas and Limitations
- Vector databases are not transactional databases. Don't use them as your source of truth. Great at similarity search, terrible at everything else databases do. Your authoritative data lives elsewhere — the vector DB is an index.
- Embedding drift is a real operational concern. Change your embedding model and all old embeddings become incompatible. You either re-embed everything (expensive) or maintain multiple indexes during a transition. Plan for this.
- Cold start is a problem for personalization. If embedding user behavior to find similar users, new users have no behavior to embed — you need fallback strategies.
- Dimensionality vs. performance is a tradeoff. Higher-dimensional embeddings capture more nuance but are slower to search and use more memory. For some applications, 128 dimensions is plenty — don't assume you need the biggest embedding your model can produce.
- Index building takes time. Building an HNSW index over 10 million vectors can take an hour or more. This affects how quickly you can deploy changes or recover from failures.
- Exact match is not what vector search does. If you need to find an exact document by ID, use a regular database. Vector search finds similar things, not identical things. Sometimes you want both.
Summary
Vector databases enable a new class of applications built on semantic similarity rather than exact matching. The core technology is approximate nearest neighbor search, with HNSW being the most common algorithm in production systems.
The practical advice is to start simple. If you're running PostgreSQL, try pgvector first. Only graduate to a purpose-built vector database when you've outgrown what extensions can provide — the complexity of operating another system is often underestimated.
In interviews, vector databases are increasingly relevant for search, recommendations, and anything involving AI/ML. Know what problem they solve (find similar things fast), how they solve it (ANN indexes like HNSW), and when they're the right tool (semantic similarity, not exact match). Start with the simple architecture and add complexity only when the requirements demand it.
The field is evolving fast — new indexing algorithms, tighter database integrations, and better tooling appear regularly. But the fundamentals of embedding data, measuring similarity, and making tradeoffs between recall, latency, and memory will remain relevant regardless of which specific technology wins.