Why an LLM trips over the r's in 'strawberry'

There’s a party trick people love: ask a powerful AI “how many r’s are in strawberry?” and watch it confidently answer “two.” It’s three. The same model can write you a working program or explain quantum mechanics, yet it fumbles a question a six-year-old gets right. That contradiction is genuinely useful, because the reason behind it explains a lot about what these systems are and aren’t.

The short version: the model never sees the letters. By the time the word reaches it, “strawberry” isn’t a string of ten characters. It’s a handful of tokens, and inside the model each token is just a number.

In what an LLM is I waved this away and said “just think word.” That’s the right simplification for understanding the big picture. This is the footnote coming back to collect, because it’s where a whole class of odd behaviour comes from.

Words go in as pieces, not letters

A model doesn’t have a slot for every possible word, there are far too many, plus typos, names, code and other languages. Instead it keeps a fixed vocabulary of maybe 100,000 to 200,000 common chunks, and builds any text out of those. Frequent words get their own token. Rare ones get assembled from smaller pieces.

So a tokeniser might cut our troublesome fruit like this:

Once it’s tokens, it’s numbers. Each token has a fixed ID, and that is what the neural network actually processes. The model has learned, from oceans of text, how these numbered chunks relate to one another, that the chunk for “berry” often follows “straw”, that “Paris” associates with “France”. What it never learned, because it never sees it, is that the “berry” chunk contains the letters b-e-r-r-y. Asking it to count r’s is like asking you how many pen strokes are in a word someone just spoke aloud. The information isn’t in the form you received.

Why chop words up at all?

It seems perverse. Why not just feed the model letters, or whole words? Both extremes are worse, and the middle ground is a genuinely clever compromise.

Sub-word tokens take the best of both. Common words stay a single efficient unit; anything unusual is still expressible by falling back to smaller pieces, right down to single characters if needed. Nothing is unspellable, and the typical sentence stays short. The standard recipe for building this vocabulary is called Byte-Pair Encoding: start from individual characters and repeatedly merge the most common neighbouring pair into a new token, until you’ve got your budget of chunks. Frequent patterns bubble up into their own tokens; rare ones stay fragmented.

The everyday consequences

This isn’t just trivia for one party trick. The token’s-eye view explains several things you’ll actually bump into.

• Letter games are hard. Counting letters, reversing words, spotting rhymes, pig latin, acrostics, anything that operates below the token, is genuinely awkward for a model, because it has to reconstruct spelling it was never directly shown. Newer models are better at it, partly through training that exposes them to the spelling, but it remains a weak spot rather than a strength.
• Maths gets fiddly. A number like 1024 might be one token, or split as “10” and “24”, and the split isn’t consistent across different numbers. Arithmetic done digit-by-digit is shaky when the model can’t reliably see the digits.
• Other languages cost more. Tokenisers are usually trained mostly on English, so English packs tightly, often a token per word. A language the tokeniser saw less of gets shattered into many small pieces, so the same sentence costs more tokens, which means it’s slower and, on paid APIs, more expensive.
• “Per-token” pricing and limits are about these. When a service charges per token or caps your context at some number of tokens, this is the unit. A rough rule of thumb for English: a token averages about four characters, so 100 tokens is roughly 75 words. Handy for estimating.

So is it stupid?

No, and that’s the interesting bit. The strawberry stumble isn’t a gap in reasoning, it’s a gap in perception. The model is reasoning perfectly well about an input that simply doesn’t contain the letters. Give it the same task in a form it can see, “spell strawberry out one letter at a time, then count the r’s”, and a capable model will often get it right, because now the letters are sitting there as separate tokens it can work with.

That’s the real lesson, and it generalises far beyond spelling: how you hand a problem to a model changes what it can do with it. Tokens are the first and most literal example of a theme that runs through everything here. The machine is powerful and weirdly shaped at the same time, and knowing the shape is most of knowing how to use it.

Next, two more places where the shape surprises people: why the same question can give you a different answer every time, and why a confident model will sometimes make things up.