Once knowledge bases exceed 1 million entries, roughly 60% of enterprise RAG projects start showing the same symptom: AI gives off-topic, incomplete, or even fabricated answers — while the underlying LLM works perfectly. Where is the problem? This week we read a comprehensive head-to-head comparison of mainstream vector databases, and the conclusion is that most enterprises built on the wrong infrastructure.
What this is
RAG (Retrieval-Augmented Generation — letting an LLM first retrieve company documents before answering) is the standard pattern for enterprise AI deployment today: split documents into chunks, convert them into "vectors" (strings of numbers representing meaning) stored in dedicated databases, then at query time find the most relevant chunks and feed them to the LLM to generate an answer.
The article offers a counterintuitive judgment: a "fast" vector database doesn't mean "accurate." The metric that truly determines answer quality is Recall@K (the proportion of truly relevant content captured in the top K results) — not the QPS (queries per second) that vendors boast about. A 3-millisecond database that only recalls 70% of relevant content is a disaster for an enterprise.
The article compares five major options — Milvus, Qdrant, Weaviate, pgvector, and Pinecone — and summarizes the most common pitfalls in enterprise selection: focusing only on QPS, ignoring P95/P99 latency (response times of the slowest requests), skipping hybrid retrieval, and omitting a Reranker (a small model that re-ranks the initial screening results).
Industry view
The pro-side verdict is straightforward: Milvus and Qdrant are far more stable than PostgreSQL's built-in pgvector for datasets in the tens of millions, hybrid retrieval, and high-concurrency scenarios. One financial knowledge base vendor measured recall jumping from 71% to 89% after switching from pgvector to Milvus.
But the dissent deserves equal airtime. An architect with deep experience in enterprise deployments stated bluntly: "Vector databases are overhyped." The reasoning: what truly determines RAG quality is the "old three" — document chunking, embedding model choice, and prompt design. Vector databases only handle the initial screening step. Other developers argue that for 90% of SMBs, pgvector plus a lightweight Reranker is sufficient — spending hundreds of thousands in annual fees on Pinecone is "using a cannon to kill a mosquito."
Impact on regular people
For enterprise IT: If your company is procuring or upgrading an AI platform, we recommend putting "recall rate" and "tail latency" into the RFP requirements — not just asking "how much concurrency does it support." These two metrics directly determine whether employees will actually use the tool.
For individual careers: AI product manager and solutions architect roles are upgrading from "understanding prompts" to "understanding the retrieval pipeline." Grasping recall and ranking logic will become hard currency in the next round of salary negotiations.
For the consumer market: No direct short-term impact. But as enterprise AI customer service and AI assistants continue improving in accuracy, the frequency of consumers encountering "customer service bots that miss the point" will visibly decline over the next 12–18 months.