# Why models make things up > A model will hand you a wrong fact, a fake citation or an invented function with total confidence. It's not lying and it's not broken. Here's where 'hallucinations' actually come from, and how to work around them. Source: https://rfriedmann.de/blog/why-models-make-things-up/ Published: 2026-06-20 · Track: learn · Level: Beginner You ask a model for a quote, a court case, a library function, a date. It gives you one, fluently, in exactly the right format, with the confident tone of something read straight off a reference shelf. Except it's wrong. The quote was never said, the case doesn't exist, the function isn't real. The unsettling part isn't the mistake, it's the *confidence*. It sounds exactly as sure as when it's right. This is usually called **hallucination**, and it's the single most important thing to understand before trusting an AI with anything that matters. The good news: once you see why it happens, it stops being mysterious, and you can work around it. ## It was never trying to be right Here's the root of it, and it's worth sitting with because it reframes everything. An LLM was not built to *retrieve facts*. It was built to **continue text plausibly**, one [token](/blog/why-models-cant-spell/) at a time, by predicting the most likely next word (see [what an LLM is](/blog/what-is-an-llm/)). That's the whole job. Truth was never the target; *plausibility* was. Most of the time those two line up. The most plausible continuation of "the capital of France is" really is "Paris", because that pairing appears constantly in its training text, so the patterns are strong and the answer is right. The trouble starts when you ask something where the plausible-sounding answer and the true answer come apart.
What the model optimises for vs. what you wanted
[diagram omitted — see the page for the chart]
The model only ever aims for the left circle: the most plausible-sounding continuation. Most of the time that overlaps with the truth. A hallucination is the model landing in the left crescent, fluent, well-formed, and wrong, with no internal signal that it's left the overlap.
A made-up citation is a perfect example. The model has seen thousands of real citations, so it knows *exactly* what one looks like: author, year, plausible title, journal, page numbers. Asked for a source it doesn't have, the most plausible continuation is a string that has all those features, a citation-shaped object. It isn't copying a real one and getting it wrong; it's generating something that fits the pattern of "citation" beautifully and happens to correspond to nothing. ## Why it sounds so sure The confidence trips everyone up, so it's worth being precise about it. A human who doesn't know something usually *sounds* unsure, they hedge, they pause. The model has no such tell, for a simple reason: fluency and accuracy are produced by the same machinery, and it's only ever good at one of them. The smoothness of the sentence is a measure of how well the words fit together, not how well they fit *reality*. A false answer can be just as grammatically smooth, just as well-structured, as a true one, so it comes out sounding identical. The model isn't suppressing doubt; from the inside there's often nothing that flags the fabrication as different from a fact. Confidence in the *prose* is not evidence about the *content*, and that decoupling is the trap. ## When it's most likely to happen Hallucinations aren't random. They cluster in predictable places, and knowing them tells you exactly when to be on guard. - **Specific facts at the edges of common knowledge.** Exact dates, statistics, prices, who-said-what, the population of a small town. The more precise and obscure, the thinner the training signal, and thin signal is where it fills in. - **Anything you ask it to cite.** Sources, page numbers, case law, study titles, quotes. Citation-shaped fabrication is a classic, precisely because the *shape* is so well learned. - **APIs, functions and config.** It'll happily invent a method name or a flag that *should* exist by the look of the rest of the library, because plausible-by-pattern is exactly what it does. - **The very recent, and the very niche.** Anything after its training cut-off, or too rare to have appeared much, it has no real basis for, so it improvises. - **When you lead it.** Ask "what's the name of the law that bans X?" and it may conjure a law to satisfy the question, even if none exists. A confident question invites a confident answer. Leave it room to say "there isn't one." ## What actually helps You can't switch it off, but you can shrink it a lot. None of this is exotic.

Invites hallucination

Keeps it honest

The single biggest lever is **give it the facts rather than asking it to recall them.** A model handed the actual document and told to answer *from that text* is far more reliable than one asked to remember, because now the plausible continuation and the true answer are forced to coincide, the right answer is sitting right there in front of it. That's the whole idea behind connecting a model to [your own documents](/blog/your-own-documents/), and it's why serious AI products lean on it so heavily. It turns "recall a fact" into "read and quote", which is a job the machine is genuinely good at. The rest is judgement. Modern models, especially the large hosted ones with web search, hallucinate far less than they did a couple of years ago, and many will now tell you when they're unsure or when a search came up empty. But "far less" is not "never", and the failure mode is invisible by design. So the rule that's served me well: **the more specific, consequential and checkable a claim is, the more it's worth checking.** Use the model to draft, to explain, to find the shape of an answer, then verify the load-bearing facts yourself. That's not distrust, it's just using the tool for what it is, a brilliant generator of plausible text, which is a different thing from an oracle. If you've read this, the [different-answers](/blog/why-different-answers/) post and the one on [tokens](/blog/why-models-cant-spell/), you've now got the three things that surprise people most about how these models behave, and most of what you need to use one without being caught out.