Andrew Ng once offered a widely cited assessment: roughly 90% of Agent projects do not fail because of model capability, but because of implementation details. RAG (retrieval-augmented generation, which retrieves information before the model answers) is one of the most fundamental components in that stack: it does not make models smarter, but helps them find more relevant material before responding.

Connecting a vector database, however, is not enough to get it right. Whether documents are chunked sensibly, relevant material is recalled accurately, results are ranked reliably, and the model uses the retrieved content faithfully all determine the final answer. If retrieval fails, even a fluent response may simply express an error with greater confidence.

What This Is

The basic RAG workflow has two stages: building the knowledge base and querying it. During indexing, an embedding model converts each passage into a sequence of numbers, or vectors; passages with similar meanings sit closer together in this “semantic space.” At query time, the system converts the question into a vector, identifies the most relevant passages, places them in the model’s context, and generates an answer based on them.

It typically involves three stages: chunking, which splits long documents into manageable passages for retrieval; recall, which retrieves semantically relevant candidates; and ranking, which places the content most likely to contain the answer first. “Semantic retrieval” therefore compares not merely whether two passages use the same words, but whether they convey similar meanings.

Industry View

We believe RAG has moved from a research concept to infrastructure for enterprise AI projects. Its value lies not in technological showmanship, but in turning internal documents, policies, product materials, and historical records into searchable knowledge entry points. For enterprise use cases where knowledge changes frequently and answers must be traceable, it is more practical than simply expanding a model’s context window.

Nevertheless, the objections are equally clear: retrieval is not fact-checking in itself. If chunking breaks semantic coherence, recall misses critical material, or ranking puts secondary information first, the system may produce an answer with citations that does not actually address the question. More problematic still, embedding models and chat models have different roles. Replacing either model while retaining the old index often creates a mismatch between the vector space and the new model.

Moreover, RAG adds the costs of document governance, access control, index updates, and result evaluation. It can improve answers, but it cannot automatically eliminate outdated materials, contradictory information, or unreliable sources. Without clear ownership and maintenance mechanisms for their data, enterprises can easily turn a knowledge base into a larger repository of confidently wrong answers.

Impact on Regular People

For enterprise IT: When companies deploy AI customer service, internal assistants, or compliance Q&A systems, the real workload shifts from training models to organizing information, configuring permissions, and continuously evaluating retrieval performance. RAG lowers the barrier to using general-purpose models, but it cannot replace foundational data governance.

For professionals: AI assistants connected to company information may increasingly explain workflows, contracts, and historical projects more accurately. Whether they are worth using depends not only on how quickly they respond, but also on whether they provide sources and recognize when the available material does not contain an answer.

For the consumer market: Products with retrieval capabilities will gradually reduce instances of models confidently fabricating answers, but consumers will still need to verify that citations correspond to the original passages and that the underlying material is current. Wider adoption does not make answers exempt from scrutiny.