Skip to main content

Hallucination, Grounding, and Uncertainty

A fluent statement unsupported by the available evidence is a system failure when the task requires grounding. The application should distinguish “not retrieved,” “conflicting,” “not authorized,” and “model failed to use evidence,” because each needs a different response.

Prerequisites​

Understand RAG stages, citation IDs, and evaluation metrics.

Learning goals​

  • verify citations against admitted context.
  • map insufficient support to a typed fallback.
  • avoid presenting token likelihood or self-reported confidence as factual certainty.

Worked example​

This literal claim checker works only for a declared fixture. It illustrates the control decision; it is not a general factuality detector.

evidence = {"kb-retention": "Expense receipts are retained for seven years."}

def decide(answer, citations, required_claim):
    citation_valid = bool(citations) and set(citations) <= evidence.keys()
    support_text = " ".join(evidence[item] for item in citations if item in evidence).lower()
    supported = citation_valid and required_claim.lower() in support_text and required_claim.lower() in answer.lower()
    return supported, citation_valid, "answer" if supported else "fallback"

good = decide("The policy says seven years.", ["kb-retention"], "seven years")
bad = decide("The policy says ten years.", ["kb-missing"], "ten years")
print(f"supported={good[0]} citation_valid={good[1]} action={good[2]}")
print(f"supported={bad[0]} citation_valid={bad[1]} action={bad[2]}")

Output:

supported=True citation_valid=True action=answer
supported=False citation_valid=False action=fallback

A useful fallback says what is missing and how to proceed: ask a clarifying question, retrieve again under the same authorization, say that evidence is insufficient, or route high-impact cases to human review. Refusal and uncertainty are product behaviors that need evaluation, not embarrassing blank outputs.

Common mistakes​

  • Calling every undesirable answer a hallucination without locating the failed stage.
  • Assuming a citation proves the cited text supports the exact claim.
  • Asking the same model to certify itself and treating that judgment as ground truth.
  • Forcing an answer when evidence is absent, stale, conflicting, or unauthorized.

Practice​

Add two sources that disagree about retention. Return conflicting_evidence with both IDs and require an owner to resolve the policy version.

Next steps​

Prompt Injection, Privacy, and Safety protects the retrieval and action boundaries around the model.

Primary sources​

Checked 2026-10-16: NIST AI RMF Generative AI Profile, FActScore, and Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.