RAG Systems: Why a Citation Is Not the Same as a Correct Answer

The evidence pipeline has several places to fail

An assistant answers a question and attaches a source link. That looks reassuring, but several questions remain. Did it retrieve the right document? Did the cited passage support the specific claim? Was a newer document available? Was the user authorized to read it? The presence of a citation answers none of these by itself. Retrieval-augmented generation, usually shortened to RAG, is best understood as an evidence pipeline whose individual stages need inspection rather than as a feature that automatically makes generated text dependable.

Google Cloud describes RAG as combining information retrieval with a language model. Google Research has also described an agentic approach that breaks down complex questions and searches iteratively for context. Those descriptions establish the general mechanism and one direction of development. They do not establish an accuracy rate for a reader’s own documents. This article examines how to assess a deployment using an original hypothetical knowledge-base example. It does not report benchmark results, quote a customer, or claim that a named vendor has solved every failure discussed here.

Start with a question that has a time boundary

Suppose an employee asks whether an international trip requires advance approval. The knowledge base contains last year’s travel policy, an updated policy, a regional exception, and an informal discussion about a proposed change. A search system can find all four because they share the same vocabulary. The useful answer depends on authority, effective date, location, and the employee’s circumstances. Semantic similarity is valuable for finding candidates, but it cannot replace the rules that determine which candidate governs the question.

This example suggests a practical document model. Keep publication date, effective date, owner, status, and applicable audience alongside the text. A document marked superseded should not compete on equal terms with the current policy. When information lacks a clear owner, the assistant should preserve that uncertainty. In some cases the best response is a concise explanation of the conflict and a link to the responsible team. Generating a confident synthesis from contradictory policies can be less helpful than admitting that the sources disagree.

Chunking is an editorial decision

A retrieval system often divides documents into smaller passages. That makes searching and fitting evidence into a model’s context more manageable, but the division can remove meaning. A paragraph stating an exception may become separated from the condition that limits it. A table can lose its column headings. A heading that identifies a country or product version may disappear when only the body is indexed. These are document-structure problems, not failures that a more persuasive prompt can reliably repair.

Review a sample of indexed passages as a reader would see them. Can each passage be interpreted with its heading and surrounding qualifications? Does it carry a stable link back to the original location? Are tables represented in a form that preserves which value belongs to which category? Test short documents as well as long manuals. A one-page policy with important footnotes can require more care than a lengthy report made of independent sections. The right unit of retrieval is the unit that preserves the claim’s meaning.

Permissions should filter the evidence first

An internal assistant may search across material that different employees can access. If restricted passages reach the model and the system merely asks it not to reveal them, the application has placed too much responsibility on the generated response. A stronger design checks authorization before returning candidate evidence to the generation stage. The authorization model also needs to account for group membership changes, document moves, and revoked access. A correct decision last week may be wrong after someone changes roles.

The answer itself can create a second disclosure path. Even when a source link is inaccessible to a recipient, a summary may reveal what the document says. Teams should therefore test whether permission changes affect cached answers, search indexes, and shared conversation histories. This is especially relevant when a user can export or forward an answer. Treating the generated text as a new copy of source information helps make the responsibility visible. A citation is attribution; it is not an access-control mechanism.

Build an evaluation set that can say no

A useful evaluation set includes questions the collection cannot answer. If every test has a convenient matching passage, an assistant can appear excellent while learning to respond confidently to everything. Include missing information, ambiguous terminology, outdated policies, and questions that require two compatible documents. Define what an acceptable refusal or request for clarification looks like. The evaluation should reward a justified statement of uncertainty rather than penalize it simply because it is shorter than an invented explanation.

Separate retrieval assessment from answer assessment. First inspect whether the correct evidence appeared among the candidates. Then inspect whether the answer represented that evidence faithfully. A failure in the first stage calls for work on indexing, search, filtering, or source quality. A failure in the second may call for clearer output constraints or a different generation approach. Keeping the stages distinct makes improvement less expensive because the team can identify the actual bottleneck instead of replacing the entire system whenever one answer disappoints.

Feedback should distinguish an incorrect answer from an incorrect source. If the assistant accurately summarizes an obsolete policy, changing the model will not fix the collection. If the correct source is current but the summary omits an exception, document maintenance alone will not fix the answer. Give reviewers a simple way to classify the issue and retain the question, retrieved passages, and generated response for investigation. Avoid including personal information in broadly shared evaluation records. A small amount of structured feedback can prevent a long series of unfocused changes.

What a trustworthy answer should show

For a practical policy answer, the visible result can be simple: the answer, the relevant conditions, the effective date, and a link to the exact supporting section. If the question spans several sources, explain how they fit together. If the evidence is incomplete, say which detail is missing. This presentation reduces the effort required for a person to verify the result. It also creates a feedback path: a reader can report an outdated section rather than merely saying that the AI was wrong.

RAG can make useful organizational information easier to reach, especially when it is scattered across many documents. Its value depends on maintaining the collection as well as improving the model. Document ownership, superseded versions, permission updates, and evidence-level evaluation are continuing responsibilities. The strongest implementation makes those responsibilities visible. A link beside an answer is a helpful starting point, but the real standard is whether the cited evidence supports the claim that the reader is being asked to trust.