Why AI hallucinates — and how to actually stop it

"AI hallucination" is the friendly name for "the model confidently said something false." Every AI product has the problem. Most marketing claims to have "solved" it. None actually have — but the gap between best-in-class and worst-in-class is huge.

Here's why it happens and what actually reduces it.

Why LLMs hallucinate

LLMs are next-token predictors. Given a prompt, they generate the most probable next token, then the next, then the next. Probability is not the same as truth. The model isn't lying — it doesn't know it's lying — it's just rolling forward on whatever pattern best matches its training.

Specific failure modes:

• Filling in gaps. Asked about something it doesn't know, the model often generates plausible-sounding text instead of refusing.
• Pattern-matching wrong patterns. "Cite a case about X" + training that includes lots of cases → confident output of an invented case name that fits the pattern.
• Confidence calibration is broken. Models don't reliably know what they don't know. They sound the same when confident and when bluffing.

What does NOT fix hallucinations

• Bigger models. GPT-4 hallucinates less than GPT-3.5, but it still hallucinates. Scale alone doesn't solve it.
• "Trust me bro" disclaimers. "AI may make mistakes" in the footer doesn't change what users do with the output.
• More training. The base problem is the architecture.

What does reduce hallucinations

1. Retrieval-augmented generation (RAG)

Give the model real source material at query time. Instruct it to answer from the retrieved chunks. Now the failure mode shifts from "invent" to "misread," which is much rarer.

2. Citation requirements

Force the model to cite its sources for every claim. The act of citing forces grounding. Combine with verification (cited chunk actually contains the claim) to catch the remaining lies.

3. Refusal training

Explicitly prompt: "If the answer is not in the retrieved chunks, say 'I don't have that information' — do not guess." This works surprisingly well.

4. Re-ranking + relevance threshold

If no retrieved chunk scores above a relevance threshold, refuse to answer. Most cases of "the model used unrelated context" go away.

5. Smaller, focused prompts

Long context dilutes attention. A 200-page document stuffed into context produces worse answers than 5 carefully-retrieved chunks. Retrieval wins on quality and cost.

6. Verification loops

For high-stakes outputs: have a second pass that checks "does the claim actually appear in the cited source?" The model often catches its own errors when asked to verify.

What SeekFiles AI does

All of the above. We publish our recall and refusal metrics in our docs because we'd rather show our work than make claims. The current state: ~93% recall on common-case retrieval, ~78% refusal-when-appropriate, single-digit invented-citation rate. Not perfect; better than ungrounded chat.

What to ask vendors

When evaluating an AI document tool, ask:

1. Do you do retrieval, or do you stuff long context?
2. Are citations chunk-level or document-level?
3. What happens when my library doesn't contain the answer?
4. Can I see the literal text you retrieved?
5. What's your hallucination rate on standard benchmarks?

A vendor that can't answer these is selling marketing. A vendor that can is building the real thing.

What you can do today

• Always click through to at least one citation per answer.
• When the answer feels surprising, ask the model where it found that — and verify.
• Use refusal-friendly prompts: "If you don't know, say so."
• Don't ask AI for facts when stakes are high and you can't verify.

Hallucinations are reduced, not eliminated. Treat AI output the way you'd treat an intern's first draft: useful, but worth checking.