Vector Databases Explained: When You Actually Need One (and When You Don't)
The vector database gold rush is in full swing. But for most teams, pgvector is all you need. Here's how to decide.
EB
Egor Burlakov
••10 min read
The vector database gold rush is in full swing, and if you are anywhere near AI or data, someone has probably told you that you “need one” for RAG. The awkward truth is that most teams do not, and that for a surprisingly large set of modern AI applications, a boring PostgreSQL instance with pgvector quietly does the job.
This post is about how to tell the difference, so you do not end up driving a Formula 1 car to the grocery store just because it looks fast.
The gold rush, and the quiet alternative
Over the past two years, vector databases have gone from niche ML tooling to a category with its own marketing budgets, conference talks, and “top ten” lists. Pinecone, Weaviate, Qdrant, Milvus, Chroma — every month another logo joins the slide.
The pitch is always seductive: “You are doing AI. You need semantic search. Therefore you need a vector database.” It sounds logical until you remember that PostgreSQL learned a few new tricks, and that the thing you are calling a “database problem” might actually be a “scale problem” you do not yet have.
Many teams are over‑optimizing for an imaginary future workload and under‑optimizing for the very real operational pain of yet another distributed system in production. The safest default is to assume you do not need a specialized vector database and force it to earn its place. The picture below introduces the framework for choosing the right solution that we are going to discuss in this blog post:
What a vector database actually is
Let us zoom out for a moment and ignore vendor names.
A vector database stores and indexes high‑dimensional vectors, which are just long lists of numbers that represent the “meaning” of some content. You feed a sentence into an embedding model like OpenAI’s text-embedding-3-small or an open‑source model such as BGE and you get a vector with hundreds or thousands of dimensions; similar sentences map to vectors that sit close together, dissimilar sentences land far apart.
The core job of a vector database is to answer one question efficiently:
Given this query vector, find the K most similar vectors in my collection.
Doing this by brute force — compare the query to every vector in your table — is fine for ten thousand vectors and unusable at a million. Approximate nearest neighbor (ANN) algorithms trade a bit of accuracy for a large speedup, and modern systems use graph‑based approaches like Hierarchical Navigable Small World (HNSW) to get something close to O(log n) search time instead of scanning every row. See a high-level example of this flow below:
Underneath the marketing, a vector database is an index and retrieval engine for ANN search. It is not a general‑purpose transactional database, and treating it as one is like buying a racehorse because you need a lawn mower.
When you genuinely need a vector database
There are three scenarios where a dedicated vector database really does earn its keep.
RAG at scale
Retrieval‑augmented generation is the biggest driver of vector database adoption today. If you are building a system that needs to search through millions of documents to give an LLM high‑quality context — a support bot over an entire knowledge base, a legal research assistant over thousands of court decisions, or a code assistant over massive repositories — you need fast vector search and good recall.
At that scale, query latency and recall quality matter; a well‑tuned HNSW or IVF index over millions of vectors, stored in a specialized system, will usually beat a general‑purpose database on both fronts, especially once you add filters, hybrid search, and frequent upserts.
Real‑time recommendation systems
The second legitimate use case is real‑time recommendation. When you recommend products, articles, or videos based on semantic similarity rather than just collaborative filtering, you are essentially running vector search on every user interaction.
“Show me visually similar products to this one,” “suggest articles related to what the user just read,” or “find songs that feel like this track” all boil down to nearest‑neighbor queries over millions of items, with strict latency budgets and continuous updates. Dedicated vector databases are designed for exactly this profile: low‑latency queries, streaming inserts, and horizontal scaling.
Multimodal search
The third growing use case is multimodal search. When you embed text, images, and audio into a single vector space and want to search across them — “find all product photos similar to this reference image and their descriptions,” for example — you quickly benefit from an engine that treats vectors as first‑class citizens.
Media companies, e‑commerce platforms, and creative tools are the obvious beneficiaries. They care about storing and querying millions or billions of embeddings, combining them with metadata filters, and tuning search quality per modality. Here, the operational overhead of a vector database is usually justified by scale.
If you live in one of these three worlds, evaluate Pinecone, Weaviate, Qdrant, Milvus, if not, you might be happier staying in Postgres land.
When you do not need one (and pgvector is quietly enough)
Now for the part that makes vendor sales teams sigh.
Small collections: under roughly 100k vectors
If your application uses fewer than one hundred thousand vectors, you almost certainly do not need a specialized vector database. At this scale, pgvector — the PostgreSQL extension for vector similarity search — is more than capable.
With an HNSW index, pgvector can serve nearest‑neighbor queries over around one hundred thousand embeddings in low double‑digit milliseconds on modest hardware, with recall typically in the mid‑90s or higher; that is fast enough for most user‑facing applications. You get vector search inside the database you likely already run in production, and you avoid adding another service to provision, monitor, back up, secure, and pay for.
If pgvector gives you ninety‑plus percent of what you need and adds zero new infrastructure, that is not a compromise; that is what good engineering looks like.
Batch analytics and offline similarity
Sometimes embeddings are not part of a real‑time retrieval system at all. You compute them for clustering, classification, anomaly detection, or other analytical tasks, and you run similarity queries as part of batch jobs.
In this world, you can store vectors in your existing data warehouse and use its native functions. BigQuery provides VECTOR_SEARCH and vector indexes, including IVF, directly in SQL so you can do semantic search and RAG over warehouse data without shipping embeddings elsewhere. Snowflake offers vector similarity functions such as VECTOR_L1_DISTANCE, VECTOR_L2_DISTANCE, and VECTOR_INNER_PRODUCT on its VECTOR type, which are designed for finding nearest neighbors inside analytics workloads.
If your embeddings live in BigQuery or Snowflake already and your retrieval is batch‑oriented, moving them into a dedicated vector database adds more moving parts than benefits.
Prototypes, POCs, and experiments
If you are building a prototype, start with the simplest thing you can possibly ship. Chroma — which positions itself as the “SQLite of vector databases” — and pgvector are perfect for early‑stage experiments, local development, and proof‑of‑concepts.
You can keep everything in a single Postgres instance or an embedded store, iterate quickly, and decide later whether you actually need a distributed vector engine. In my experience, most POCs never reach the scale that would justify a specialized vector database, or they pivot into a direction where the original choice no longer matters.
The major players, briefly
If you have convinced yourself that you really do need a vector database, here is the short version of the landscape.
Pinecone is the fully managed option. You send vectors, it stores, indexes, and serves them. Operational overhead is minimal, and pricing at scale reflects that. It is ideal for teams that would happily pay a premium to avoid running their own indexes in production.
Weaviate offers self‑hosted and managed deployments, with strong multimodal support and built‑in pipelines for vectorization. It is more feature‑rich than Pinecone, which is another way of saying it is also more complex.
Qdrant has become a popular Rust‑based engine with excellent performance, sensible defaults, and a generous open‑source tier. It is attractive for teams that want to self‑host without inheriting the operational complexity of very large‑scale systems such as Milvus.
Milvus targets massive scale — billions of vectors, enterprise installations, and complex topologies. It is powerful and correspondingly demanding to operate; unless your workload truly lives in the “billions of embeddings” world, it is likely more than you need.
Chroma stays lightweight and developer‑friendly, aiming to be the embedded, local, and prototyping option rather than your forever production store.
You can spend days comparing benchmark charts and feature matrices, but in practice, most teams pick between “do we stay on pgvector” and “do we move to Pinecone or Qdrant” and call it done.
A simple decision framework
When someone asks whether they need a vector database, they should answer three deliberately boring questions as boring questions tend to lead to sane architectures.
1. How many vectors are you searching?
Numbers come first because they constrain everything else.
Under roughly 100k vectors, use pgvector and do not think twice; HNSW indexes in Postgres handle this range with good recall and single‑digit to low double‑digit millisecond latency on ordinary hardware.
Between 100k and 10M, pgvector can still work with careful indexing and tuning, but if low single‑digit millisecond latency is critical to your user experience, it is worth evaluating a specialized solution.
Above 10M, a proper vector database is usually worth the operational overhead, especially if you care about filters, hybrid search, and frequent updates at that scale.
Pick your range honestly, based on what you have or will have in the next year, not based on deckware about “billions of users.”
2. What is your latency requirement?
Next, decide how much you care about speed in absolute terms.
If you need sub‑10ms retrieval for real‑time applications serving end users — for example, recommendations that run on every page view or semantic search on every keystroke — a dedicated vector engine with optimized in‑memory indexes is often the right choice.
If you can tolerate 50–100ms latency, or your queries are batch‑oriented, pgvector or warehouse‑native functions will do fine and will be easier to operate. Most customer support bots, internal tools, and analytical jobs live in this more forgiving zone, even if they will never admit it.
3. How much operational complexity can your team absorb?
Finally, and most importantly, look at your team.
Every new piece of infrastructure has a maintenance cost that goes far beyond the monthly bill. Someone has to monitor it, patch it, back it up, secure it, and debug it at three in the morning. If your team is already busy keeping your existing stack upright, adding a distributed vector database cluster may hurt more than it helps.
The best architecture is the one your team can actually operate. A perfectly chosen vector database that is poorly maintained is worse than pgvector that is well monitored and regularly backed up. Starting with the simplest option and upgrading only when you hit a real, measured limit is not being conservative; it is respecting your future self.
The boring answer that saves you time
So do you actually need a vector database?
If you are running RAG or recommendations at genuine scale, with millions of embeddings and strict latency budgets, dedicated vector systems are a powerful tool and you should absolutely consider them. If you are building smaller AI features, prototypes, internal tools, or batch analytics, you can probably do everything you need with pgvector, BigQuery, Snowflake, or Chroma, and you will sleep better at night for having one fewer distributed system in your stack.
And the next time a vendor tells you that every application with an LLM needs a dedicated vector database, smile politely, think about your vector counts and your latency budget, and remember that when you are a hammer, everything looks like a nail — but you do not have to volunteer to be the nail.
EB
Written by Egor Burlakov
Engineering and Science Leader with experience building scalable data infrastructure, data pipelines and science applications. Sharing insights about data tools, architecture patterns, and best practices.
Explore Further
Dive deeper into the tools and categories mentioned in this article.
