Vector Databases: Fast Similarity Search over Embeddings

An embedding can represent the meaning of one item, but an application may need to compare a query against millions of stored vectors.

Scanning every vector for every request becomes expensive.

A vector database stores vectors and uses specialized indexes to retrieve nearby vectors efficiently, usually together with identifiers and metadata.

It answers similarity questions. It does not replace the databases responsible for exact records, transactions, accounting, or relational integrity.

A concrete example: approved support articles

A support assistant receives the question “Why was my card payment reversed?”

The application:

1. embeds the question,
2. filters the index to the user's region and active documents,
3. finds nearby support chunks,
4. returns their text and source metadata,
5. and gives those results to a response system.

The original content remains in governed document storage. The vector database provides fast candidate retrieval.

A basic vector record

A stored record commonly contains:

• a unique identifier,
• the embedding vector,
• source text or a reference to it,
• document and chunk identifiers,
• timestamps,
• category or language,
• tenant and permission fields,
• and version information.

The identifier connects similarity results back to authoritative data.

Exact nearest-neighbour search

The simplest search compares the query vector with every stored vector and sorts by similarity.

This can provide exact nearest neighbours, but work grows with collection size and vector dimension. It may be acceptable for small datasets, offline analysis, or a filtered subset.

Large interactive systems often use an approximate index.

Approximate nearest neighbours

Approximate nearest-neighbour, or ANN, indexes avoid scanning everything. They organize vectors so the search explores promising regions.

Common families include:

• graph-based indexes such as HNSW,
• inverted-file approaches,
• tree-like structures,
• and compressed vector techniques.

They trade some recall for lower latency and resource use. “Approximate” means the index may miss a true nearest item.

The indexing tradeoff

Index settings affect:

• build time,
• memory,
• disk use,
• query latency,
• update cost,
• and retrieval recall.

Searching more candidates generally improves recall but consumes more computation. Tune with representative production queries rather than a synthetic benchmark alone.

Metadata filtering

Similarity should operate inside valid boundaries.

A company assistant may need to filter by:

• tenant,
• user permissions,
• country,
• product version,
• language,
• publication status,
• or effective date.

Filtering after retrieval can be both inefficient and unsafe. Prefer systems that apply security-relevant filters during candidate selection, and verify that every returned source remains authorized.

Vector databases and ordinary databases

Use a transactional database for:

• account balances,
• orders,
• uniqueness constraints,
• multi-record transactions,
• and exact state changes.

Use vector retrieval for:

• semantically similar documents,
• related products,
• duplicate reports,
• or candidate examples.

A production system often uses both. The ordinary database is the system of record; the vector index is a derived search structure that can be rebuilt.

Vector databases and keyword search

Traditional search engines offer inverted indexes, exact term matching, phrase search, filtering, and mature ranking features. Vector systems offer semantic neighbourhoods.

Hybrid retrieval combines keyword and vector candidates, then merges or reranks them. An error code such as E1047 needs exact matching, while “I cannot access my account” benefits from semantic matching.

Ingestion and synchronization

Indexing is a pipeline:

1. detect a source change,
2. parse and clean the content,
3. divide it into meaningful chunks,
4. compute embeddings,
5. attach metadata,
6. upsert records,
7. and remove obsolete versions.

The system must handle partial failure. If content updates but embeddings do not, search can serve stale text. Track checkpoints, retries, dead-letter records, and freshness.

Deletes matter

Deleting a source record should remove every derived chunk and vector. This matters for:

• privacy requests,
• expired contracts,
• revoked permissions,
• corrected guidance,
• and tenant deletion.

Use deterministic identifiers or source-to-chunk manifests so the system can find all derivatives. Periodic reconciliation should compare the index with authoritative storage.

Updating an embedding model

A new embedding model usually requires a new index because its vectors occupy a different space.

Run the migration as a versioned rollout:

• re-embed a representative evaluation collection,
• compare retrieval quality,
• build the new index,
• dual-read or shadow-test,
• switch traffic,
• and retire the old index after rollback risk passes.

Do not overwrite vectors gradually inside one incompatible index.

Replication and partitioning

At scale, records may be partitioned across machines and replicated for availability.

Partitioning can follow tenant, region, or a vector-space strategy. The choice affects query fan-out, balancing, locality, and isolation. Replication improves read availability but requires a plan for index freshness and consistency.

Measure the whole retrieval path

Monitor:

• index recall on labelled queries,
• end-to-end answer usefulness,
• p50 and tail latency,
• filter selectivity,
• stale-record rate,
• indexing delay,
• failed upserts and deletes,
• and cost per query.

A fast vector lookup is not success if ingestion is stale or the best result is filtered out incorrectly.

Security boundaries

Treat vector search as a data access path. Enforce authentication, authorization, tenant isolation, encryption, audit logging, and rate limits.

Never assume that semantic obscurity protects content. If a user can issue queries and receive text, the system is exposing information through retrieval.

Choosing a system

Consider:

• collection size and growth,
• vector dimension,
• update frequency,
• filtering requirements,
• target latency and recall,
• hybrid-search needs,
• operational expertise,
• regional deployment,
• backup and recovery,
• and portability.

For modest workloads, an existing database extension may be enough. A specialized platform is justified when measured scale or features require it.

Knowledge check

1. Why is exhaustive vector search expensive at scale?
2. What tradeoff does an ANN index make?
3. Why should permission filters be applied during retrieval?
4. Which data should remain in a transactional system of record?
5. What steps are needed when changing embedding models?

The one idea to remember

Vector databases make nearest-neighbour search practical through specialized indexes and metadata filtering. They are derived retrieval systems whose quality depends on ingestion, synchronization, permissions, evaluation, and integration with authoritative databases.