The Bottom Line
- Hallucinations are best managed as a systems risk: define what ‘major’ vs ‘minor’ means, test it, then train safe behaviour.
- Pressure-test any AI tool with adversarial prompts, boundary conditions, and ‘I don’t know’ expectations.
- A credible evaluation should be explicit about error definitions and human review—don’t accept vague reassurance.
Healthcare isn’t the place for ‘close enough’. Hallucinations matter because they can create false confidence. The most robust approach is not fear; it’s discipline: define error severity, run structured tests, and train clinicians to verify high-impact claims. Recent UK-facing evaluation work has explicitly simulated hallucination behaviours and categorised ‘major’ hallucinations in clinical contexts—use that mindset even when you’re evaluating commercial tools.
Pressure-test checklist (use on any AI clinical search tool)
1
1) Boundary prompts
Ask the tool questions slightly outside its intended scope and observe behaviour. A safe tool should either refuse or clearly state uncertainty—not invent confident detail.
2
2) Citation stress test
Ask for the source behind the strongest claim and open it. If sources are not accessible or don’t support the claim, downgrade trust immediately.
3
3) Consistency test
Re-ask the same question with minor phrasing changes. Large swings in answers without explanation are a risk signal.
4
4) ‘I don’t know’ expectations
Tools that claim ‘no hallucinations’ or similar should explicitly disclose what happens when evidence is absent. A good system will stop rather than guess.
5
5) Human factors
Train a habit: if the output would materially change an action, verify the primary source. This is safety behaviour, not mistrust.
Use UK-style evaluation framing
Prefer evaluation language that defines ‘major’ vs ‘minor’ hallucinations and uses structured review, rather than marketing reassurance. This is how you make safety operational.
SourceExample: UK hallucination simulation framing (MHRA report PDF)
Open Link SourcePraktiki: ‘no hallucinations’ claim (official)
Open Link