# How a 30-billion-parameter model fits on one card

> Quantisation, explained for normal people: how shrinking each number in a model lets a giant fit on a desktop graphics card, and what it costs.

Source: https://rfriedmann.de/blog/how-a-big-model-fits-on-one-card/
Published: 2026-04-10 · Track: learn · Level: Beginner

A model like Qwen3-30B has 30 billion numbers inside it (its [weights](/blog/what-is-an-llm/)).
Stored the normal way, that is about 60 gigabytes. An RTX 5090 graphics card holds
32. The numbers don't add up, and yet the model runs on that card just fine. The
trick that makes it possible has an ugly name and a simple idea.

<aside class="defbox"><span class="tag">In plain words</span><b>Quantisation:</b> storing each of a model's numbers using fewer bits, so the whole thing takes up less space (and runs faster), in exchange for a tiny loss of precision.</aside>

## A number doesn't need 16 bits

By default, each weight is stored using 16 bits, which is 2 bytes. That's roomy:
it can represent a number very precisely. But a model's weights don't need that
much precision. You can store roughly the same value in 8 bits, or even 4.

<figure class="chart">
<div class="chart-title">The same number, stored in fewer bits</div>
[diagram omitted — see the page for the chart]
<figcaption class="chart-caption">Each square is one bit. Fewer bits per number means a smaller file, at the cost of a little precision.</figcaption>
</figure>

Fewer bits means a rougher version of each number, but also a much smaller model.
And it turns out "rough but close" is good enough for the billions of weights in a
language model.

## Why it matters: it fits, and it's faster

Shrink every number from 16 bits to 4 and the whole model shrinks with it.

<figure class="chart">
<div class="chart-title">A 30-billion-parameter model: file size by precision</div>
[diagram omitted — see the page for the chart]
<figcaption class="chart-caption">FP16 won't fit. FP8 only just fits, with no room to actually run. FP4 fits with space to spare for everything else.</figcaption>
</figure>

That buys you two things at once. First, the model actually **fits** in the card's
memory, with room to spare for the [working memory it needs while it
runs](/blog/what-is-a-gpu/). Second, it runs **faster**: generating text means
hauling all those weights through the chip for every word, and fewer bytes means
less hauling. (That speed story has [its own
post](/blog/serving-30b-models-rtx-5090/).)

## The catch: precision

There's no free lunch. Fewer bits means each number is less exact, and push it too
far and the model gets noticeably dumber.

The honest way to picture it is a JPEG photo. Save it smaller and it loses a little
detail you'll probably never notice; crush it too hard and you see ugly blocks.
Quantisation is the same trade. Crude 4-bit hurts; but modern 4-bit formats are
clever about it (they keep a little extra scaling information for each small group
of numbers), and the result stays remarkably close to the full-size model.

That last trick is exactly what [imp](https://github.com/kekzl/imp) leans on: it
converts a model to 4-bit when it loads, which is the whole reason a 30-billion
parameter model runs on a desktop card at all. If you want the hardware side of
why that card can and can't do certain things, see [what a 5090 is missing next to
a data centre GPU](/blog/what-the-5090-lacks-vs-datacenter/).