# How an image generator creates a picture from noise > An image generator neither paints nor assembles a collage: the model starts from random noise and removes it step by step, steered by your words. How that works, and why it explains the distorted hands. Source: https://rfriedmann.de/blog/how-image-generators-work/ Published: 2026-11-08 · Track: learn · Level: Beginner The [previous lesson](/blog/how-models-see-images/) went one way: picture in, words out. An image generator goes the other way, words in, picture out, and it would be natural to guess it works like a chatbot run backwards, painting the image one patch at a time. It doesn't. The engine behind almost every image generator is a different family, the [diffusion model](/blog/ai-that-isnt-an-llm/), and its approach sounds backwards the first time you hear it: it learns to make pictures by learning to *remove noise*. ## Training: ruin pictures, then learn to undo it Training starts with millions of images and their captions. Take one, add a little random noise, then a little more, and keep going until nothing is left but static. The original recipe (Ho et al., 2020, the "DDPM" paper) used 1,000 such steps from clean to pure noise. Now the model gets one job: shown a noisy picture, the step number and the caption, predict the noise that was added. That's all it ever learns. It never sees "draw a cat" as an instruction; it sees thousands of half-ruined cat photos next to the word "cat" and gets very good at guessing which speckles don't belong. That skill is enough to generate images. Hand the trained model a canvas of *pure* noise and the caption "a red house on a hill". It guesses the noise, you subtract a bit of it, and the result is slightly less random and slightly more house-like. Feed that back in. After enough rounds there is nothing random left, only a house nobody photographed.
Generation: the same network, run again and again on its own output
[diagram omitted — see the page for the chart]
A toy version with 36 pixels. A real model does the same on a whole image at once: every step looks at the entire canvas, so the picture sharpens everywhere together instead of being drawn left to right like a sentence.
## How the words get a say The prompt reaches the model through a **text encoder**, a language model that turns your sentence into the same kind of meaning-vectors the [vision lesson](/blog/how-models-see-images/) described. At every step the denoiser looks at those vectors while it decides what to remove. On its own that steering is weak, so nearly every generator adds a technique called **classifier-free guidance** (Ho and Salimans, 2022). At each step the model makes two guesses, one with your prompt and one with an empty prompt, and the final step is pushed further in the direction of the difference. A setting usually called the *guidance scale* says how hard. Turn it up and the picture follows your words more literally, and past a point it gets harsh and oversaturated. It also means two passes per step, which is part of why generation takes a while. ## The shortcut that made it practical: work small Denoising a full photo pixel by pixel is expensive. The development that put this on ordinary graphics cards was **latent diffusion** (Rombach et al., 2022, the paper behind Stable Diffusion). A separate small network, an autoencoder, first squeezes the image into a compact grid of numbers, the *latent*, and all the noising and denoising happens in there. Only at the very end is the result decoded back into pixels. The saving is large. Stable Diffusion's first versions turned a 512 x 512 image with 3 colour channels (786,432 numbers) into a 64 x 64 grid with 4 channels (16,384 numbers): 48 times fewer values to push through every step. ## What changed since: transformers and straighter paths The first generation used a convolutional network as the denoiser. Two ideas replaced it in the current models: - **Diffusion transformers.** Peebles and Xie (2022) swapped the denoiser for the same transformer design that powers LLMs, working on patches of the latent. Like LLMs, it got reliably better the bigger it was made. - **Flow matching.** Instead of learning to remove noise along a wiggly path, the model learns a straight route from noise to image (Lipman et al., 2022; Stable Diffusion 3 in 2024 scaled up a close cousin, "rectified flow"). Straighter routes need fewer steps. Fewer steps is the thing you feel. A *distilled* model is trained to cover the route in a handful of big steps instead of dozens of small ones. Alibaba's Z-Image-Turbo, for example, is a 6-billion-parameter model distilled to 8 steps. Fewer steps means faster pictures, usually paid for with a little less variety between attempts. ## Why hands and text used to come out wrong Early generators often produced six-fingered hands and signs in unreadable script. Both had the same root: the model only learns what it can see clearly and often in its training data, and it reasons about the picture through a blurry compressed latent. Hands are small in most photos, partly hidden, and in endlessly varied poses, so the model learned "skin-coloured fingery region" rather than "exactly five fingers". Text was worse: the early text encoders were small and treated your prompt more like a bag of ideas than a spelled-out string, so the model knew *a sign* was wanted but not which letters were on it. Current models use far larger denoisers, much better captions in training, and a full language model as the text encoder. Alibaba's Qwen-Image, a 20-billion-parameter diffusion transformer from 2025, was built specifically to render text in images, including Chinese. Hands and short words now usually come out right; long paragraphs and exact counts are still where to look first for mistakes. ## Editing is the same process with a mask Inpainting, "replace the sky", "remove the person on the left", is not a separate technology. You paint a mask, the area outside it is held fixed, and the denoising steps run only inside it, each step blending in with the untouched pixels around it. Image-to-image works the same way: instead of pure noise, start from your own picture with some noise added, and the model cleans it up towards the new prompt. The less noise you add, the more of the original survives. ## Running one yourself Image models are much smaller than chat models, which makes them a good first local experiment after [your first local model](/blog/your-first-local-model/). The arithmetic for the weights alone, at 2 bytes per parameter in 16-bit and 1 byte in 8-bit: | Model | Size | 16-bit weights | 8-bit weights | Licence | |---|---|---|---|---| | FLUX 3 Image Klein (Black Forest Labs, Oct 2026) | 5.6 B | 5.6 x 2 = 11.2 GB | 5.6 GB | FLUX Kommunity License, revenue limit | | Z-Image-Turbo (Alibaba, Nov 2025) | 6 B | 6 x 2 = 12 GB | 6 GB | Apache 2.0 | | Qwen-Image (Alibaba, Aug 2025) | 20 B | 20 x 2 = 40 GB | 20 GB | Apache 2.0 | These are estimates for the denoiser only; the text encoder and the autoencoder come on top. The licence column matters as much as the size column: "open weights" can still mean revenue limits or conditions on commercial use, so read it the way the [licence lesson](/blog/open-weights-and-licenses/) describes before anything goes into a product. ## Provenance of generated images Generated images raise two questions an IT team will get asked. **Who owns it** is its own topic, covered in [who owns AI output](/blog/who-owns-ai-output/). **Can anyone tell it's generated** is where provenance comes in. The C2PA standard, shown to users as *Content Credentials*, attaches a signed record to a file saying which tool made it and how it was edited. It is useful and fragile at the same time: many platforms strip metadata on upload, and a missing credential proves nothing either way. In the EU the [AI Act](/blog/ai-and-the-law-gdpr-ai-act/) adds a legal side: providers of generators must mark output as machine-generated in a machine-readable way, and deployers must disclose deepfakes. ## The takeaway An image generator is not a painter and not a collage machine. It is a noise remover that got good enough to remove noise that was never covering anything, steered at every step by your words. That one idea explains its strengths, a whole image refined at once, and its old weaknesses, details it never saw clearly. Everything since, latents, transformers, straighter paths, fewer steps, is engineering on top of that one idea.