# How an image generator creates a picture from noise
> An image generator neither paints nor assembles a collage: the model starts from random noise and removes it step by step, steered by your words. How that works, and why it explains the distorted hands.
Source: https://rfriedmann.de/blog/how-image-generators-work/
Published: 2026-11-08 · Track: learn · Level: Beginner
The [previous lesson](/blog/how-models-see-images/) went one way: picture in, words
out. An image generator goes the other way, words in, picture out, and it would be
natural to guess it works like a chatbot run backwards, painting the image one patch
at a time. It doesn't. The engine behind almost every image generator is a different
family, the [diffusion model](/blog/ai-that-isnt-an-llm/), and its approach sounds
backwards the first time you hear it: it learns to make pictures by learning to
*remove noise*.
## Training: ruin pictures, then learn to undo it
Training starts with millions of images and their captions. Take one, add a little
random noise, then a little more, and keep going until nothing is left but static.
The original recipe (Ho et al., 2020, the "DDPM" paper) used 1,000 such steps from
clean to pure noise.
Now the model gets one job: shown a noisy picture, the step number and the caption,
predict the noise that was added. That's all it ever learns. It never sees "draw a
cat" as an instruction; it sees thousands of half-ruined cat photos next to the word
"cat" and gets very good at guessing which speckles don't belong.
That skill is enough to generate images. Hand the trained model a
canvas of *pure* noise and the caption "a red house on a hill". It guesses the noise,
you subtract a bit of it, and the result is slightly less random and slightly more
house-like. Feed that back in. After enough rounds there is nothing random left,
only a house nobody photographed.
Generation: the same network, run again and again on its own output
[diagram omitted — see the page for the chart]
A toy version with 36 pixels. A real model does the same on a whole image at once: every step looks at the entire canvas, so the picture sharpens everywhere together instead of being drawn left to right like a sentence.
## How the words get a say
The prompt reaches the model through a **text encoder**, a language model that turns
your sentence into the same kind of meaning-vectors the [vision
lesson](/blog/how-models-see-images/) described. At every step the denoiser looks at
those vectors while it decides what to remove.
On its own that steering is weak, so nearly every generator adds a technique called
**classifier-free guidance** (Ho and Salimans, 2022). At each step the model makes two
guesses, one with your prompt and one with an empty prompt, and the final step is
pushed further in the direction of the difference. A setting usually called the
*guidance scale* says how hard. Turn it up and the picture follows your words more
literally, and past a point it gets harsh and oversaturated. It also means two passes
per step, which is part of why generation takes a while.
## The shortcut that made it practical: work small
Denoising a full photo pixel by pixel is expensive. The development that put this on
ordinary graphics cards was **latent diffusion** (Rombach et al., 2022, the paper behind
Stable Diffusion). A separate small network, an autoencoder, first squeezes the image
into a compact grid of numbers, the *latent*, and all the noising and denoising
happens in there. Only at the very end is the result decoded back into pixels.
The saving is large. Stable Diffusion's first versions turned a 512 x 512 image with
3 colour channels (786,432 numbers) into a 64 x 64 grid with 4 channels (16,384
numbers): 48 times fewer values to push through every step.
## What changed since: transformers and straighter paths
The first generation used a convolutional network as the denoiser. Two ideas replaced
it in the current models:
- **Diffusion transformers.** Peebles and Xie (2022) swapped the denoiser for the same
transformer design that powers LLMs, working on patches of the latent. Like LLMs,
it got reliably better the bigger it was made.
- **Flow matching.** Instead of learning to remove noise along a wiggly path, the model
learns a straight route from noise to image (Lipman et al., 2022; Stable Diffusion 3 in 2024
scaled up a close cousin, "rectified flow"). Straighter routes need fewer steps.
Fewer steps is the thing you feel. A *distilled* model is trained to cover the route in
a handful of big steps instead of dozens of small ones. Alibaba's Z-Image-Turbo, for
example, is a 6-billion-parameter model distilled to 8 steps. Fewer steps means
faster pictures, usually paid for with a little less variety between attempts.
## Why hands and text used to come out wrong
Early generators often produced six-fingered hands and signs in unreadable
script. Both had the same root: the model only learns what it can see
clearly and often in its training data, and it reasons about the picture through a
blurry compressed latent.
Hands are small in most photos, partly hidden, and in endlessly varied poses, so the
model learned "skin-coloured fingery region" rather than "exactly five fingers".
Text was worse: the early text encoders were small and treated your prompt more like
a bag of ideas than a spelled-out string, so the model knew *a sign* was wanted but
not which letters were on it. Current models use far larger denoisers, much better
captions in training, and a full language model as the text encoder. Alibaba's
Qwen-Image, a 20-billion-parameter diffusion transformer from 2025, was built
specifically to render text in images, including Chinese. Hands and short words now
usually come out right; long paragraphs and exact counts are still where to look
first for mistakes.
## Editing is the same process with a mask
Inpainting, "replace the sky", "remove the person on the left", is not a separate
technology. You paint a mask, the area outside it is held fixed, and the denoising
steps run only inside it, each step blending in with the untouched pixels around it.
Image-to-image works the same way: instead of pure noise, start from your own picture
with some noise added, and the model cleans it up towards the new prompt. The less
noise you add, the more of the original survives.
## Running one yourself
Image models are much smaller than chat models, which makes them a good first local
experiment after [your first local model](/blog/your-first-local-model/). The arithmetic
for the weights alone, at 2 bytes per parameter in 16-bit and 1 byte in 8-bit:
| Model | Size | 16-bit weights | 8-bit weights | Licence |
|---|---|---|---|---|
| FLUX 3 Image Klein (Black Forest Labs, Oct 2026) | 5.6 B | 5.6 x 2 = 11.2 GB | 5.6 GB | FLUX Kommunity License, revenue limit |
| Z-Image-Turbo (Alibaba, Nov 2025) | 6 B | 6 x 2 = 12 GB | 6 GB | Apache 2.0 |
| Qwen-Image (Alibaba, Aug 2025) | 20 B | 20 x 2 = 40 GB | 20 GB | Apache 2.0 |
These are estimates for the denoiser only; the text encoder and the autoencoder come
on top. The licence column matters as much as the size column: "open weights" can still
mean revenue limits or conditions on commercial use, so read it the way the [licence
lesson](/blog/open-weights-and-licenses/) describes before anything goes into a product.
## Provenance of generated images
Generated images raise two questions an IT team will get asked. **Who owns it** is its
own topic, covered in [who owns AI output](/blog/who-owns-ai-output/). **Can anyone tell
it's generated** is where provenance comes in. The C2PA standard, shown to users as
*Content Credentials*, attaches a signed record to a file saying which tool made it and
how it was edited. It is useful and fragile at the same time: many platforms strip
metadata on upload, and a missing credential proves nothing either way. In the EU the
[AI Act](/blog/ai-and-the-law-gdpr-ai-act/) adds a legal side: providers of generators
must mark output as machine-generated in a machine-readable way, and deployers must
disclose deepfakes.
## The takeaway
An image generator is not a painter and not a collage machine. It is a noise remover
that got good enough to remove noise that was never covering anything, steered at every
step by your words. That one idea explains its strengths, a whole image refined at
once, and its old weaknesses, details it never saw clearly. Everything since, latents,
transformers, straighter paths, fewer steps, is engineering on top of that one idea.