Q&A

What is hallucination in AI/LLMs?

By AlgoInsight Team2026-06-18

Overview

An LLM doesn't have a concept of truth - it models the probability distribution of text. When asked about something it doesn't know well, it doesn't say "I don't know" by default; it generates the most statistically plausible continuation, which can be confidently wrong. This is the core hallucination problem.

graph TD
    A["LLM asked: 'What did Einstein say about quantum mechanics?'"] --> B["Model generates plausible-sounding quote"]
    B --> C{Is it real?}
    C -->|Often not!| D["Hallucination: fluent + confident + false"]
    C -->|Sometimes| E["Correct recall from training data"]
    style D fill:#6366f1,color:#fff
    style A fill:#1e1b4b,color:#fff

Types of Hallucination

Factual hallucination

The model states incorrect facts, often with high confidence:

• Fabricating citations, paper titles, or authors that don't exist.
• Making up statistics or dates.
• Attributing quotes to real people who never said them.

Faithfulness hallucination

The model contradicts or goes beyond the provided context:

• Summarizing a document but adding facts not in the original.
• Extracting information from a passage that isn't there.

Reasoning hallucination

The model makes logical errors in multi-step reasoning, especially with maths or symbolic tasks.

Why LLMs Hallucinate

1. Next-token prediction objective - the model learns what text looks like, not what is true.
2. No world model - LLMs don't maintain a persistent, verifiable model of reality.
3. Training data limitations - gaps, outdated facts, and noise in training data propagate to outputs.
4. Overconfident priors - the model "fills in the blanks" from strong statistical associations even when uncertain.
5. Rare knowledge - the less frequent a fact appears in training data, the more likely the model is to confabulate around it.

Mitigation Strategies

Retrieval-Augmented Generation (RAG)

Ground responses in retrieved documents. Instruct the model: "Answer using only the provided context. If the answer isn't in the context, say so."

Constrained prompting

"Answer only what you are certain about. If you are not sure, say 'I don't know'
rather than guessing. Cite the specific part of the context you used."

Temperature reduction

Lower temperature (e.g. 0.0–0.3) makes outputs more deterministic and factual, reducing creative hallucination - but doesn't eliminate it.

Self-consistency

Generate multiple responses at higher temperature; take the majority-vote answer. Reduces variance in reasoning tasks.

Chain-of-thought + verification

Ask the model to reason step by step, then verify its own answer:

"Solve the problem. Then check your work by substituting the answer back in."

Tool use

Let the model call a web search, calculator, or database instead of relying on parametric memory for facts it might get wrong.

Model calibration and output confidence

Some fine-tuning approaches train models to express calibrated uncertainty ("I'm not sure, but...") rather than false confidence.

Human-in-the-loop / factual verification

For high-stakes outputs, route uncertain responses to human review or an automated fact-checking step.

Evaluating Hallucination

• TruthfulQA - benchmark of questions LLMs commonly answer incorrectly.
• FActScore - measures factual precision of generated biographies.
• LLM-as-judge - use a separate LLM to verify factual claims in generated text.
• Human evaluation - gold standard for measuring faithfulness to source documents.

RAG Doesn't Fully Solve It

Even with RAG, models can hallucinate by ignoring the retrieved context, combining context with incorrect parametric knowledge, or misattributing retrieved facts. Good RAG requires instructing the model explicitly to stay within the provided context and to signal uncertainty.

Why This Matters in Interviews

Hallucination is one of the most important practical limitations of LLMs. Interviewers want you to explain: why it happens (next-token prediction, no truth model), the types (factual, faithfulness, reasoning), and the mitigation strategies (RAG, constrained prompting, tool use, CoT + verification). The honest framing - "it's a fundamental property, not just a bug" - and knowing that no single technique eliminates it shows maturity.

Common Pitfalls

• Thinking RAG eliminates hallucination - it reduces it but doesn't remove it.
• Treating lower temperature as a reliable fix - it helps but the model can still hallucinate.
• Over-trusting confident-sounding outputs - confidence and accuracy are poorly correlated in LLMs.
• Not testing for hallucination on your specific domain - general benchmarks may not reflect your use case.

References

• Ji et al., "Survey of Hallucination in Natural Language Generation" (2022): https://arxiv.org/abs/2202.03629
• TruthfulQA benchmark: https://github.com/sylinrl/TruthfulQA

More answers