They’re not hallucinations…
There’s something about the term “hallucination” when applied to a large language model’s wrong but confident answer that disagrees with me. Let’s unpack.
If a person hallucinates, they’re in an altered mental state. Maybe from drugs. Maybe from hunger, thirst, or sleep deprivation. Maybe from a traumatic brain injury. It’s a disruption to the normal workings of their mind that causes them to think or hear or see something that’s not there.
If an LLM hallucinates, it’s not at all due to damage or tampering with their internal structure. When ChatGPT confidently mentions books that don’t exist, for example, it’s not because someone took a wrench to OpenAI’s server banks or let a virus loose on their code.
Here’s a better analogy. Imagine you’re in English class, called on by the teacher to state your opinion about a Shakespearean sonnet you were supposed to read. You didn’t do the reading, so you just say: “Ah yes, I really liked it; I liked how it felt meaningful without ever quite saying why, like Shakespeare was hinting at something emotional that I was definitely almost understanding the whole time.” That’s not a hallucination, it’s a plausible-sounding answer to the teacher’s question. A non-answer, to be exact, because it’s not grounded in reading we should have done.
It might sound nitpicky to obsess over terminology, but the mental models and analogies we use inform how we think deeper about things. The “hallucination” view implies a temporary deficiency that we can overcome or avoid, whereas the “non-answer” view implies that we get such non-answers every time the model is out of its depth, like the student who didn’t do the assigned reading.
With that mental model, the way to avoid, or at least catch, non-answers is to pose questions in such a way that non-answers are not a plausible way to continue our exchange. Part of that is prompt and context engineering:
Don’t assume that a model knows facts. That’s how you end up hearing about books that don’t actually exist.
Include relevant content directly in the prompt OR
Provide access to a trusted knowledge base via tools such as the model context protocol (MCP) or retrieval-augmented generation (RAG))
Offer a graceful backdown. LLM-based chatbots are trained to be helpful, so “I don’t know” does not come naturally for them.
We don’t have to get ChatGPT off LSD or shrooms to get correct answers; we have to know what questions even make sense to ask, and what context to provide.
