What Is a Vector Database? Meaning, Pronunciation, and How It Works

What Is a Vector Database?

A vector database is a specialized database designed to store high-dimensional numerical vectors, called embeddings, and to retrieve them by similarity rather than by exact match. Embeddings are generated from text, images, audio, video, or any other content by running the input through an embedding model such as OpenAI text-embedding-3-small, Cohere embed-v3, or an open-source alternative like BGE. The resulting vector — typically several hundred to a few thousand floating-point numbers — encodes the semantic meaning of the input in a form that computers can compare quickly. A vector database indexes millions or billions of such vectors and returns the ones nearest to a given query vector in milliseconds.

The familiar analogy is the difference between a traditional library catalog and a knowledgeable librarian. A relational database is the catalog: it excels at pinpointing a book when you know its exact identifier, but it cannot help when you ask for “something like this.” A vector database is the librarian: it understands the meaning of your request and offers you the closest matching titles, even when your wording does not literally appear in any of them. This shift from exact-match search to semantic search is the reason vector databases became essential infrastructure for generative AI applications starting in 2023 — and why they are now one of the fastest-growing database categories in the industry.

Although the mathematics behind vector search is decades old, productization only accelerated once high-quality embedding models became cheap and accessible. Before 2020, producing good semantic embeddings required bespoke training pipelines; today any developer can call an API and get a usable embedding in under 100 milliseconds, which dramatically lowers the barrier to building semantic-search features. Keep in mind that the category is still maturing, so feature sets, pricing, and operational maturity vary significantly between vendors.

How to Pronounce Vector Database

How a Vector Database Works

The heart of any vector database is two concepts: embeddings and approximate nearest neighbor (ANN) search. Content is first turned into a vector by an embedding model, and that vector is stored in the database along with metadata. At query time, the search input is embedded by the same model, and the database returns the stored vectors with the smallest cosine distance or Euclidean distance to the query. Note that exact nearest-neighbor search is possible but prohibitively expensive past a few thousand items, which is why ANN indexes are used in practice.

ANN algorithms

HNSW (Hierarchical Navigable Small World), IVF (Inverted File Index), and PQ (Product Quantization) are the three dominant ANN families. HNSW offers an excellent speed-accuracy tradeoff and is the default in most modern vector databases, which is why you will encounter it most often in documentation and tutorials. IVF partitions the vector space for faster filtering, and PQ compresses vectors to reduce memory usage. Keep in mind that the “right” algorithm depends on your corpus size, dimensionality, and latency budget.

Dimensionality reduction and quantization

Storing raw 1536-dimensional floats at 4 bytes each means ~6 KB per vector, so 100 million vectors consume about 600 GB of memory. Production deployments therefore use PQ or binary quantization to shrink vectors by 10× to 32×, with only modest accuracy loss. This is an important optimization to understand because it is often what separates a prototype from a scalable product.

Metadata and hybrid search

Real applications rarely search by vector alone. You typically want to filter by tenant, date range, tags, or permissions. Modern vector databases support metadata indexes alongside vector indexes so you can combine “semantic similarity” with traditional filters in a single query. Some products go further and expose first-class hybrid search that fuses BM25 keyword scores with vector similarity, which tends to outperform either approach alone.

Vector Database Usage and Examples

The most common real-world pattern is retrieval-augmented generation (RAG): embed your documents, store them in a vector database, and at query time retrieve the most relevant chunks for the LLM to ground its answer. The code below shows the minimal implementation with pgvector, a PostgreSQL extension that has become the most popular vector database by installed base in 2026 because it reuses an existing database instead of adding a new system.

-- Enable extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Create table (1536 dims = text-embedding-3-small)
CREATE TABLE docs (
  id SERIAL PRIMARY KEY,
  content TEXT,
  embedding VECTOR(1536)
);

-- HNSW index for cosine distance
CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);

-- Insert
INSERT INTO docs (content, embedding)
VALUES ('A vector DB excels at similarity search', '[0.123, -0.456, ...]');

-- Retrieve top-5 nearest neighbors
SELECT id, content
FROM docs
ORDER BY embedding <=> '[0.11, -0.44, ...]'
LIMIT 5;

The `<=>` operator computes cosine distance; lower values mean higher similarity. In application code, you would normally call this from psycopg2, asyncpg, or an ORM like SQLAlchemy, passing the query embedding from the same model you used at ingestion time.

Popular vector databases

Advantages and Disadvantages of Vector Databases

Advantages

• Enables semantic search. Synonyms, paraphrases, typos, and even cross-language queries are handled naturally by the embedding space.
• Foundation for RAG. Lets LLMs cite up-to-date, private, or company-specific knowledge that is not in their training data.
• Multimodal support. Images, audio, and video can live in the same vector space with compatible embedding models.
• Scales to billions. Properly tuned ANN indexes deliver sub-100ms latency on enormous corpora, which is important for consumer-scale products.

Disadvantages

• Storage and memory cost. Without quantization, large corpora quickly become expensive. You should budget for compression from day one.
• Embedding lock-in. Changing embedding model requires re-embedding the entire corpus. Plan for this migration cost when you pick a provider.
• Quality is not automatic. Chunking, overlap, top-K, and reranking all influence result quality. Expect iterative tuning.
• Operational complexity. Running Milvus or Qdrant yourself is nontrivial. If you lack DevOps bandwidth, managed products often win.

Vector Database vs RDBMS vs Full-Text Search

Three categories of database, three different notions of “match.” The comparison below is worth internalizing because it shows up in nearly every architecture discussion once you move beyond a prototype. Note that modern stacks frequently use all three in the same request.

Common Misconceptions

Misconception 1: A vector database alone solves RAG quality

It does not. Retrieval quality depends at least as much on chunking strategy, reranking, and prompt design as on the database itself. Switching from Pinecone to Milvus rarely moves the needle unless you change the surrounding pipeline. Keep in mind that vector search is a necessary component, not a sufficient one.

Misconception 2: A vector database is an entirely separate system

pgvector and the vector types in MongoDB, Redis, and Elasticsearch show that vector search is increasingly a feature of general-purpose data stores. Evaluate whether you actually need a separate product before procuring one.

Misconception 3: More dimensions is always better

Past a certain point, extra dimensions increase cost without improving retrieval. For many tasks, a well-tuned 384-dim or 768-dim embedding performs as well as a 3072-dim one. Note that dimension reduction is an active area of research, and many production teams use models like Matryoshka embeddings to trade accuracy for speed dynamically.

Real-World Use Cases

Enterprise RAG

Internal documentation, runbooks, policies, and past tickets are embedded and indexed so the company chatbot can answer employee questions with cited sources. ChatGPT Enterprise, Claude for Enterprise, and Glean all rely on vector databases as their retrieval backbone.

Product recommendation

E-commerce platforms embed product descriptions and user behaviors to surface semantically similar items. This has gradually replaced classical collaborative-filter recommenders at many large retailers.

Image and video search

Using CLIP or similar multimodal models, a vector DB can answer “find me pictures that look like this” and “find near-duplicate uploads.” The approach is widely used in content moderation and digital asset management.

Customer support automation

Past tickets are embedded so that when a new ticket arrives, the system surfaces the most similar resolved cases and a draft reply. Support teams report significant handle-time reductions.

Fraud and anomaly detection

Transaction sequences or user-behavior vectors are compared against known fraudulent patterns. The interesting thing about this use case is that it shows vector search is not exclusively about language — it generalizes to any signal you can embed.

Semantic code search

GitHub Copilot, Sourcegraph Cody, and many IDE plugins embed code snippets so that developers can find “the function that parses a CSV” even when they do not remember the function name. Code embeddings are trained differently from text embeddings, but the database layer is the same.

Drug discovery and bioinformatics

Molecular structures and protein sequences can be embedded with domain-specific models such as ESM or ChemBERT. Pharmaceutical companies store these vectors to search large compound libraries for candidates similar to a promising lead. This is an important example because it shows vector search reaching high-value, non-text domains.

Design Considerations for Production

Getting a vector search demo running is easy; running a dependable production service is much harder. Several design decisions recur across deployments and are worth planning for up front rather than retrofitting later.

Chunking strategy

You cannot embed an entire 50-page document as a single vector and expect good recall. Documents are split into chunks of a few hundred tokens, often with some overlap so that semantic units are not cut in half. The right chunk size depends on the content; technical documentation often works well at 300–500 tokens, while narrative text may benefit from larger chunks. Keep in mind that chunking is the single biggest lever on retrieval quality in most RAG pipelines.

Reranking

Vector search is fast but not the most accurate way to rank candidates. A common pattern is to retrieve the top 50 candidates with vector search and then rerank them with a smaller cross-encoder model (Cohere Rerank, BGE Reranker, Voyage Rerank). This two-stage approach is cheap compared to calling a reranker on the entire corpus, yet it sharply improves precision@10.

Metadata filtering

In multi-tenant products, every query must filter by tenant ID. In enterprise search, permission filters keep users from seeing documents they should not. Implementing this correctly — at index time, not after retrieval — is critical to both correctness and performance. Most mature vector databases expose a filter syntax that pushes predicates down into the index.

Refresh and update patterns

Documents change. You need a strategy for deleting, updating, and reindexing vectors when underlying content changes. Some products handle this efficiently; others effectively require rebuilding an index. Ask about update costs during evaluation, because they dominate operational burden more than initial ingest speed.

Monitoring retrieval quality

Retrieval is not a one-time setup; it drifts as content, queries, and embedding models evolve. Production teams keep a labeled evaluation set of representative queries and grade top-K retrieval quality on every change. Without this, degradations go unnoticed until users complain. Note that “LLM-as-judge” graders are widely used for cost reasons, but they have their own failure modes and should be spot-checked by humans.

Cost budgeting

Storage, compute for ANN queries, bandwidth, and calls to the embedding model all contribute to the bill. Build a cost model early. For example, if you expect one million active documents at 768 dims with int8 quantization, that is ~750 MB of raw vectors plus overhead — manageable in-memory on a single node. Ten million at 3072 dims with full float32 is a very different conversation.

Embedding Model Selection

The database is only half the story. The quality of your search ultimately depends on the embedding model. Here is a short tour of the main options as they stand in 2026, along with the tradeoffs each one brings.

OpenAI text-embedding-3 family

A safe general-purpose default for English. The small variant returns 1536-dim vectors at very low cost; the large variant returns 3072 dims and supports Matryoshka truncation for runtime flexibility. Integration is trivial via the OpenAI API, and quality is consistently strong on public benchmarks. Keep in mind that pricing and rate limits apply to both ingest and query time.

Cohere embed-v3 and embed-v4

Strong multilingual support and optional compression modes. Particularly popular in enterprise settings where non-English content is important. The Cohere ecosystem also includes best-in-class reranker models that pair naturally with their embedder.

Voyage AI

A focused vendor that has, as of 2026, topped leaderboards on retrieval tasks for domain-specific embeddings (legal, code, finance). If you have a specialized corpus, it is worth evaluating against generic models because the gap can be surprisingly large.

Open-source options

BGE, E5, Nomic Embed, and the SentenceTransformers zoo give you models you can run locally or self-host on your own GPUs. This matters for regulated industries, air-gapped deployments, and cases where the data you embed is too sensitive to ship to an API provider. The tradeoff is that you take on operational responsibility for the model.

Specialist embeddings

CLIP and its successors embed images and text in the same space, enabling multimodal search. CodeBERT, StarEncoder, and Voyage Code embed source code with awareness of syntax and identifiers. Protein language models like ESM embed biological sequences. The broader point is that “embedding” is now a spectrum of tools, not a single commodity.

Benchmarking and Evaluation

When you compare vector databases, synthetic benchmarks rarely predict real performance. Evaluate on your own corpus with queries that resemble what users will actually ask. The MTEB benchmark is a useful starting point for embedding model selection, and ANN-benchmarks is a good reference for index algorithms, but neither replaces a task-specific evaluation.

A simple but effective evaluation pattern is the following: collect 50–200 real queries, label the 1–3 most relevant documents for each, and measure recall@10. Track this number over time as you change the model, chunking, or index parameters. If it moves the wrong direction, you have a regression before users see it. Note that maintaining the labeled set is itself a non-trivial ongoing cost, but it is the only way to have confidence in retrieval quality over the long term.

Future Trends

Several trends are reshaping the vector database space as of 2026. First, traditional databases are absorbing vector features: Oracle, MongoDB, Redis, Elasticsearch, DynamoDB, and even SQLite now offer vector search as a first-class capability. This makes the “dedicated vector database” category less distinct than it was two years ago, and for many workloads the boring choice of “add vectors to the database you already run” is the right one.

Second, embedding quality continues to improve at an impressive rate. Matryoshka embeddings let applications choose dimensionality at query time, trading storage and latency against accuracy. Learned quantization approaches narrow the gap between compressed and uncompressed recall. Multimodal embeddings that jointly embed text, images, tables, and audio are becoming practical. Keep in mind that this means your embedding strategy deserves periodic review rather than one-time choice.

Third, retrieval and generation are blurring. Late-interaction models like ColBERT, generative retrieval, and differentiable search indexes hint at future architectures where the database and the language model are coupled more tightly. It is too early to call the winners, but engineers building new systems should assume the architectural landscape five years from now will look meaningfully different from today. You should be prepared to revisit design decisions as the ecosystem evolves.

A fourth trend worth watching is graph-augmented retrieval. Combining vector similarity with explicit graph structure — for example, citation graphs in academic search or entity relationships in enterprise knowledge graphs — consistently improves retrieval for complex queries. Products like Neo4j with vector indexing and Microsoft GraphRAG exemplify this direction, and it is likely that many production RAG systems in the coming years will add a graph layer on top of their vector layer. Note that while pure vector search works for the average query, the hardest queries often benefit from additional structure.

Frequently Asked Questions (FAQ)

Q1. Which vector database should I choose?

If you already run PostgreSQL, start with pgvector. If you prefer a managed service, Pinecone is the shortest path. For self-hosted, open-source, high-scale, Milvus or Qdrant are both solid. For quick Python experimentation, Chroma is ideal. Keep in mind that your choice is reversible early on but becomes sticky once you have billions of embeddings.

Q2. Do I need hybrid search?

Probably yes. Combining BM25 keyword search with vector similarity consistently beats either method alone on real benchmarks. Most mature vendors now ship first-class hybrid search; if yours does not, consider how you will layer it yourself.

Q3. Which embedding model should I use?

For English general use, OpenAI text-embedding-3-small and Cohere embed-v3 are safe defaults. For multilingual support, Cohere multilingual or BGE M3 are strong. If you need on-premise, E5, BGE, or Nomic Embed are proven open-source options.

Q4. How big does my corpus need to be to justify a vector DB?

Anything above a few thousand documents benefits. Below that, a brute-force Python dot-product over an in-memory array is often faster and cheaper. Above a few million, you almost certainly want a purpose-built index.

Conclusion

• A vector database stores embeddings and retrieves them by similarity using ANN indexes.
• Core technologies: embedding models and ANN algorithms like HNSW, IVF, and PQ.
• The foundation of RAG, enabling LLMs to cite private, current, or domain-specific knowledge.
• Popular products: Pinecone, Weaviate, Milvus, Qdrant, pgvector, Chroma.
• Works best when combined with keyword search, metadata filters, and reranking in a hybrid pipeline.
• Selection depends on scale, operational model, and whether you already run PostgreSQL.
• Retrieval quality depends on chunking, reranking, and prompt design — not just the database.

References