Evaluate LLM and RAG systems with cases you can defend

Write the expected behavior before the answer exists

Each case should include a question, relevant document IDs, required facts, forbidden claims, and whether clarification or abstention is appropriate. Add policy versions and access scope where relevant. Include paraphrases, typos, exceptions, unsupported topics, conflicting documents, and unauthorized data requests.

Keep a development set for iteration and a held-out set for release comparison. If you constantly rewrite prompts after seeing a particular test, that test is becoming development data. Human authors should check reference answers against the actual sources; a model-generated answer is not automatically ground truth.

Measure retrieval with a worked example

Suppose two documents are relevant and retrieval returns three: one relevant document and two irrelevant ones. Recall@3 is 1/2, or 0.5. Precision@3 is 1/3. If the first relevant result is ranked second, reciprocal rank is 1/2. Average reciprocal rank across questions gives MRR.

relevant = {"returns", "damage"}
retrieved = ["delivery", "returns", "catalog"]
hits = len(relevant & set(retrieved))
recall = hits / len(relevant)
precision = hits / len(retrieved)
rr = next((1 / rank for rank, doc in enumerate(retrieved, 1)
           if doc in relevant), 0.0)
print(round(recall, 3), round(precision, 3), round(rr, 3))
# 0.5 0.333 0.5

Deduplicate results before interpreting these simple metrics. For questions with no relevant document, define an abstention measure instead of dividing by zero or pretending ordinary recall applies.

Score the answer separately

Check whether the answer addresses the question, preserves required facts, avoids unsupported claims, cites supporting sources, and follows the requested format. Groundedness asks whether claims follow from the supplied evidence; factual correctness may also require checking whether that evidence is itself accurate and current.

Schema checks and allowed-source checks can be automated deterministically. Semantic judgments may use a rubric with human reviewers or a calibrated model judge. A judge can favor verbosity, miss subtle errors, or be influenced by the text it evaluates. Sample its decisions against human review and record disagreement. BLEU or ROUGE can describe text overlap, but a paraphrase may be correct with low overlap and a nearly identical sentence may reverse a crucial condition.

Make release decisions from several signals

Track latency distributions rather than only averages: the slowest tail often shapes user frustration. Record cost per completed task, error rates, unnecessary retries, and abstention quality. A faster system that fabricates more answers has not necessarily improved.

Compare versions on the same cases and inspect regressions individually. Small datasets have substantial uncertainty; report counts alongside percentages. For example, 9/10 is informative but does not prove a stable 90% production accuracy. Keep production feedback separate from sensitive raw conversation storage and turn confirmed failures into reviewed future cases.

References and further reading

Use these primary references for deeper study and current API details. Examples in this lesson use fictional Northstar data.

• Official OpenAI documentation — model-optimization