Hallucination (AI)
When an AI model generates text that sounds confident and plausible but is factually incorrect — a fundamental limitation of large language models that matters enormously in research contexts.
Hallucination refers to the tendency of large language models (LLMs) to generate false information with the same fluency and apparent confidence as true information. The term comes from the observation that the model “sees” something that isn’t there — producing a fabricated citation, a made-up statistic, or an incorrect fact stated as if it were established.
Hallucinations occur because LLMs are trained to produce plausible-sounding text, not to retrieve verified facts. The model has no internal mechanism for distinguishing what it knows from what it’s confabulating. It will invent a paper citation with a real-sounding author, journal name, and year — and the citation simply won’t exist.
Why it matters for research:
- Literature citations are a particularly dangerous hallucination type — LLMs frequently generate plausible but nonexistent references. Always verify any citation an LLM produces against a real database (PubMed, Google Scholar, Semantic Scholar).
- Clinical and pharmacological facts — drug dosages, interactions, and guidelines — must never be sourced from an ungrounded LLM response.
- Quantitative claims — effect sizes, p-values, sample sizes — are frequently hallucinated when an LLM summarizes research it hasn’t actually read.
How to reduce hallucination risk:
- Use tools that retrieve and cite specific papers rather than generating text from memory — Consensus, Elicit, and Semantic Scholar are grounded in retrieved literature.
- Use retrieval-augmented generation (RAG) approaches where the model is given the actual document text to work from, rather than relying on training data memory.
- Treat any LLM factual claim as a hypothesis to verify, not a finding to cite.
Related terms: Retrieval-Augmented Generation (RAG), Prompt Engineering
Related guides: Health & Medicine, Social Science