Hallucination detection
Hallucination detection is the practice of finding unsupported or contradicted claims in generated output. In production, the practical question is usually groundedness: are the answer's claims supported by the context the system was allowed to use?
What counts as a hallucination
- Unsupported claim: the answer states something not present in the evidence.
- Contradicted claim: the answer conflicts with the evidence.
- Fabricated citation: the answer cites a source, section, or URL that does not support it.
- Overgeneralization: the answer expands a narrow fact into a broad conclusion.
- Missing uncertainty: the answer should say the evidence is incomplete but presents certainty.
Detection approaches
| Approach | Works when | Weakness |
|---|---|---|
| Claim extraction plus evidence check | answers have separable factual claims | can miss implicit claims |
| LLM judge for groundedness | evidence is provided and rubric is clear | judge may over-trust fluent text |
| Citation verification | citations are required | citations can be relevant but insufficient |
| Human review | stakes are high or ambiguous | expensive and slower |
Design for detectability
- Require citations or quoted evidence for factual answers.
- Keep retrieved context attached to traces.
- Ask the model to state uncertainty when evidence is missing.
- Separate "answer quality" from "evidence support" in the eval rubric.
Pitfall
Do not treat hallucination as a mysterious model property only. Many hallucinations in RAG products are system failures: poor retrieval, missing context, ambiguous prompt, or UI that forces an answer when the right behavior is to abstain.
Connects to: why LLMs hallucinate · grounding and citations · RAG system eval