# What a GPU is, and why AI needs one

> Why running AI means buying a graphics card, what makes a GPU different from a CPU, and why the amount of memory on the card is the number that really matters.

Source: https://rfriedmann.de/blog/what-is-a-gpu/
Published: 2026-04-03 · Track: learn · Level: Beginner

To run AI at home, everyone tells you to get a "GPU", a graphics card. Which is a
little odd, because you're not playing a video game. So why does generating text
need the same chip that draws explosions in games?

<aside class="defbox"><span class="tag">In plain words</span><b>GPU (graphics card):</b> a chip built to do a huge number of simple calculations at the same time. Originally for drawing game graphics, now the workhorse of AI.</aside>

## A few big workers, or thousands of small ones

The chip you already have, the **CPU**, has a handful of very powerful cores. It's
brilliant at doing one complicated thing after another, quickly. A GPU is the
opposite bet: thousands of much simpler cores that all work at the same time.

<figure class="chart">
<div class="chart-title">Why AI loves a GPU</div>
[diagram omitted — see the page for the chart]
<figcaption class="chart-caption">AI is the same simple sum done across billions of numbers. A CPU does a few at a time, fast; a GPU does thousands at once. That parallelism is the whole reason GPUs run AI.</figcaption>
</figure>

That suits AI perfectly. Underneath all the cleverness, running a model is the
same humble sum, multiply two numbers and add, repeated across billions of
weights for every word. A CPU grinds through those a few at a time. A GPU does
thousands at once. That's the whole reason AI moved onto graphics cards.

## The number that really matters: memory

Raw speed isn't the catch, though. The catch is memory.

<aside class="defbox"><span class="tag">In plain words</span><b>VRAM:</b> the graphics card's own built-in memory. The model has to fit in here to run fast. A model that doesn't fit either crawls or won't load at all.</aside>

An RTX 5090 has 32 gigabytes of VRAM. The model's weights, plus the [working
memory it needs while it runs](/blog/how-a-big-model-fits-on-one-card/), all
have to live inside that budget.

<figure class="chart">
<div class="chart-title">What fills a 32 GB card (one RTX 5090)</div>
[diagram omitted — see the page for the chart]
<figcaption class="chart-caption">The model has to fit in here. Quantization is what shrinks it enough to leave room for the working memory it needs while it runs.</figcaption>
</figure>

So when people compare graphics cards for AI, two numbers do most of the talking.
**How much** memory (that decides how big a model you can run) and **how fast**
that memory is (that decides how quickly it writes each word). On a 5090 those are
32 GB and about 1.79 terabytes per second.

This is also why [quantisation](/blog/how-a-big-model-fits-on-one-card/), the
trick of shrinking the model, matters so much: it's what gets a big model under
that 32 GB ceiling. And if you want to know what this particular card can and
can't do compared to the giant ones in data centres, that's [its own
story](/blog/what-the-5090-lacks-vs-datacenter/).