Most RAG Tutorials Assume Your Data Sits Still. Business Data Doesn't.

The standard RAG tutorial goes like this. Take your documents. Split them into chunks. Turn each chunk into a vector. Store the vectors. When a question arrives, embed the question, find the nearest chunks, hand them to the model as context.

That pipeline is correct, and it works well for exactly one kind of problem: a body of knowledge that does not change much. Product documentation, policy manuals, research archives, a knowledge base.

Now consider the questions people actually ask a business system.

What is the status of my order? Do you have this in stock in my size? What did I spend on this account last quarter? Is this appointment slot available?

None of these have answers that live in a document. They live in a database or behind an API, and the answer changed this morning — possibly since the user opened the page. Embedding them is not merely inefficient. It is incoherent, because there is no stable text to embed.

This is the gap between RAG as taught and RAG as deployed.

Two Different Architectures Wearing One Name

Static corpus RAG is the tutorial version. Documents are ingested ahead of time, chunked, embedded, and indexed. Retrieval is a similarity search over that index. The engineering problems are chunking strategy, embedding quality, and getting the right passages into the top results.

Dynamic data RAG works differently. There is no pre-built index of the answer, because the answer does not exist until you ask. The system has to interpret the question, decide which system holds the answer, construct a query against a live source, get a structured result back, and give that to the model as context.

The retrieval step is not a vector search. It is a routing and query-construction problem — closer to translating a question into an API call than to finding similar text.

Most real applications need both, and this is the part that gets underestimated. A customer support system needs the policy documents and the customer's actual order. A financial assistant needs the explanatory material and the current balance. Building only the first half produces a system that answers general questions well and specific questions confidently wrong, which is worse than not answering at all.

Where Static RAG Actually Breaks

Even when your data genuinely is documents, the tutorial pipeline has failure modes that only show up with real content.

Chunking destroys context. Split a document at a fixed size and you will cut tables in half, separate a heading from what it introduces, and strip the sentence that says "this only applies to enterprise customers" from the paragraph it qualifies. The retrieved chunk then reads as a general rule. The model has no way to know something was removed, and neither does the user.

Similarity is not relevance. Vector search finds text that resembles the question. Sometimes that is the answer; sometimes it is a passage that discusses the same topic without containing the fact. A question about refund eligibility retrieves five paragraphs about refunds, none of which state the eligibility rule. Semantically close, practically useless.

Keywords still matter. Pure semantic search is unreliable for exact identifiers — part numbers, error codes, proper names, version strings. Someone searching for a specific SKU wants that exact string, and embeddings are bad at exact. This is why hybrid approaches, combining keyword and vector search, outperform either alone on real query mixes.

Nothing tells you retrieval failed. This is the quiet one. If retrieval returns nothing useful, the model does not stop. It produces a fluent answer from whatever it has, which is its general knowledge, and the output looks identical to a well-grounded one. There is no visible failure — just an answer that is wrong for this customer.

That last point is the one worth building around, because it means retrieval quality is invisible from the output. You cannot evaluate a RAG system by reading its answers. You have to evaluate the retrieval step separately, on its own, against known-correct expectations.

What Production Actually Requires

Beyond the pipeline, the things that consume the schedule.

Query routing. Deciding whether a question needs documents, live data, both, or neither. Get this wrong and everything downstream is wrong.

Freshness policy. For dynamic sources, how stale is acceptable? Caching an inventory count for an hour is a business decision with a cost attached, not a technical detail.

Access control. The single most under-designed part of most RAG systems. Retrieval must respect who is asking. A vector index built over all internal documents, queried without permission filtering, is a mechanism for exposing anything anyone ever wrote to anyone who phrases a question well. This has to be enforced at retrieval, not by asking the model to be discreet.

Latency budget. Embedding, searching, calling APIs, then generating — each step adds time, and the sum determines whether the experience is usable. Streaming the response helps enormously with perceived speed, which is why it belongs in the design rather than being retrofitted.

Testing that targets retrieval. Known questions with known correct sources, checked automatically, so that a change to chunking or embeddings does not silently degrade recall. Without this you will not find out about a regression until a customer does.

The Short Version

RAG is not one technique. It is a category, and the tutorial covers the easiest member of it.

If your data is documents that rarely change, the standard pipeline is genuinely most of the work. If your users ask about their own current state — orders, balances, availability, anything with a timestamp — you are building something else, and the retrieval layer is a routing problem rather than a search problem.

Building the wrong one produces a system that demos beautifully on the FAQ and fails on the first real question.

Retrieval Augmented Generation with Node.js covers both architectures — embeddings, in-memory and database-backed vector storage, dedicated vector databases, dynamic RAG against external and internal APIs, hybrid search, streaming responses, and the performance, security, testing, and deployment work that follows.