Build a RAG pipeline that can say “the policy does not cover that”

Build two paths, not one giant function

The ingestion path loads approved sources, extracts usable text, divides it into passages, attaches metadata, computes embeddings, and updates an index. The answering path validates a request, retrieves permitted passages, assembles context, generates an answer, and checks the result.

Separating these paths lets you investigate failures. If a policy never entered the index, changing the answer prompt will not fix it. If retrieval found the correct paragraph but the answer ignored an exception, rebuilding the index may be unnecessary. Keep enough trace information to identify the layer without logging sensitive content indiscriminately.

Chunk around meaning and boundaries

Start with section-aware chunks: keep a heading with its paragraph and preserve exceptions next to the rule they qualify. A fixed token size with overlap is a baseline to test, not a universal setting. A table row without its column headings may be meaningless; a return window without exclusions may be misleading.

For the three tiny Northstar policies, one record per policy is sufficient. Longer documents may benefit from searching a focused child passage and then supplying its parent section. Record source IDs and positions during ingestion. Trying to reconstruct provenance from generated text afterward is fragile.

Assemble a bounded answer request

The outline below is pseudocode: each named function represents application behavior you must implement. It shows where the evidence and authorization checks belong, rather than pretending a framework call supplies them automatically.

identity = authenticate(request)
query = validate_question(request.question)
candidates = retrieve(query, allowed_scope=identity.scope)
passages = verify_access_and_versions(candidates, identity)
context = select_relevant_passages_within_budget(passages)
result = generate_policy_answer(query, context)
check_source_ids(result, allowed_ids=context.ids)
review_or_measure_claim_support(result, context)
return render_answer_or_insufficient_evidence(result)

Source-ID validation is deterministic. Claim support is a harder evaluation problem and may require rules, model-assisted review, or human sampling depending on the application. Do not label a response verified merely because its source URL exists.

Make unsupported cases useful

Ask “Can I return a clearance item after 10 days?” The return policy supports explaining the exclusion. Ask “When will my bank receive the refund?” None of the three documents provides a timeline, so the assistant should say what is missing and offer an appropriate next step.

Contradictory versions need an explicit policy for effective dates and authority. If your system cannot establish which applies, surface the conflict. RAG supplies external context; it does not eliminate hallucinations or turn every retrieved passage into truth. Evaluate source quality, retrieval quality, and answer quality separately.

References and further reading

Use these primary references for deeper study and current API details. Examples in this lesson use fictional Northstar data.

• Research paper — 2005.11401