“Wouldn’t it be cheaper to run it ourselves?” comes up every time an API invoice lands on my desk. I build imp, an inference engine for one RTX 5090, so I have the local numbers at hand. This post puts both sides into the same unit, euros per million tokens, with every input sourced and the arithmetic shown.
What you are paying for#
Both sides bill in tokens. A token is a piece of text, often a whole common word, sometimes a fragment of a rarer one. It is the unit the model reads and writes, which is also why models can’t spell: they never see the letters.
API prices, as of November 2026#
I picked one model per tier from the official pricing pages of Anthropic and OpenAI, standard tier, no batch discount, no caching:
| Model | Tier | Input, USD / M | Output, USD / M | Output, EUR / M |
|---|---|---|---|---|
| Claude Fable 5.2 | frontier | 8.00 | 40.00 | 36.32 |
| Claude Sonnet 5.5 | mid | 2.00 | 10.00 | 9.08 |
| gpt-6.1-terra | mid, cheaper | 1.00 | 5.00 | 4.54 |
| Claude Haiku 5.5 | small | 0.10 | 0.50 | 0.45 |
Haiku 5.5 is priced for prompts up to 100 000 tokens; above that it costs 0.50 USD input and 2.50 USD output. The euro column uses the ECB reference rate of 1.1014 USD per euro on 16 November 2026.
Why output costs five times as much#
In every row above, output costs exactly five times input. That follows from how the hardware works. Your prompt is processed in one go: thousands of tokens pass through the model in parallel, and the GPU stays busy calculating. The answer comes out one token at a time, and for each token the GPU reads the weights it needs from memory again. That second phase is limited by memory bandwidth, not by arithmetic, so a generated token occupies the hardware far longer than a read one. I wrote that out in detail in prefill vs decode.
For the comparison below I use output prices. Input is cheaper per token but there is usually much more of it, so a real bill depends on your prompt-to-answer ratio.
The local side: inputs#
| Input | Value | Source |
|---|---|---|
| RTX 5090, cheapest offers in Germany | about 7 700 EUR | Notebookcheck, 14 Nov 2026 |
| Household electricity, Germany 2026 | 37.3 ct/kWh | BDEW, October 2026 |
| Energy, one stream | 0.115 Wh per 500-token answer | my measurement, 17 Nov 2026 |
| Energy, 32 streams | 0.024 Wh per 500-token answer | my measurement, 17 Nov 2026 |
| Idle draw | 39 to 49 W, 49 W used here | my two measurements |
| Decode, one stream | about 350 tok/s, 30B-class MoE in NVFP4 | imp |
| Decode, 32 streams | 1 975 tok/s aggregate, Qwen3.8-27B | imp |
Card and electricity prices include 19% VAT. A company that reclaims VAT and pays a commercial electricity rate pays less, so the local figures below are an upper bound.
Yes, 7 700 EUR for a consumer card. It is the largest number in this calculation. I count the card only: no host machine, no setup, no admin time, all of which would make local more expensive.
The arithmetic#
Fixed cost per year. I write the card off over 3 years: 7 700 / 3 = 2 566.67 EUR. The machine runs as a server, always on. Idle draw is 49 W (the higher of my two readings), so for every hour it is not generating it costs 49 Wh.
Energy per million output tokens. One stream: 0.115 Wh per 500 tokens is 0.23 kWh per million tokens, at 37.3 ct/kWh that is 0.086 EUR. 32 streams: 0.024 Wh per 500 tokens, because the card’s power draw barely rises while its output grows fivefold. That is 0.047 kWh or 0.018 EUR per million tokens.
Scenario A: one stream, busy a full working year. 2 000 hours of generation (8 hours on 250 days) at 350 tok/s is 2 520 million tokens. Idle for the remaining 6 760 hours costs 331 kWh, which is 123.55 EUR. Fixed cost: 2 566.67 + 123.55 = 2 690.22 EUR, divided by 2 520 million tokens is 1.07 EUR, plus 0.09 EUR energy: 1.15 EUR per million.
Scenario B: 32 streams, same 2 000 hours. 1 975 tok/s for 2 000 hours is 14 220 million tokens. 2 690.22 / 14 220 = 0.19 EUR, plus 0.02 EUR energy: 0.21 EUR per million. Each of the 32 users then sees about 62 tok/s, which is still fast enough to read along.
Scenario C: one person, normal use. Scenario A assumes someone keeps the card generating eight hours a day, nonstop. Nobody reads that much. Take a person who reads 200 answers of 500 tokens per working day: 100 000 tokens a day, 25 million a year. The card generates for about 20 hours a year and idles the rest, so the fixed cost is 2 726.41 EUR. Divided by 25 million tokens: 109.14 EUR per million. That is three times the frontier API price.
Break-even against each API#
Local has a large fixed cost and a tiny marginal cost, the API has no fixed cost and a fixed price per token. The break-even volume is the yearly fixed cost divided by the difference per million tokens. With 2 690.22 EUR fixed and the batched energy cost of 0.018 EUR per million:
| Against | Break-even, million output tokens / year | Per working day |
|---|---|---|
| Claude Fable 5.2 | 74 | 296 000 |
| Claude Sonnet 5.5 | 297 | 1.19 million |
| gpt-6.1-terra | 595 | 2.38 million |
| Claude Haiku 5.5 | 6 165 | 24.7 million |
Against Fable 5.2 the card pays off at about 600 answers of 500 tokens per working day. Against Haiku 5.5 it needs 6 165 million tokens a year, more than one stream can produce in 2 000 hours (2 520 million). Only a card that keeps many requests in flight at once, through continuous batching, reaches that volume.
The catch: it is not the same model#
The table compares price, not results. A 30B-class open model on one card is not a frontier model, and the 74-million break-even against Fable 5.2 assumes the local model does the same job, which it usually doesn’t. I argued in self-hosting an LLM for your team that the error rate of a weaker model costs more working time than the API fees it saves. The fair price comparison for a model this size is the small tier, and against Haiku 5.5 local only wins when 32 streams are busy most of the working day.
Two more things move the numbers. API prices change: Google’s Gemini pricing page, for example, lists prices that double for some models on 1 December 2026 or 1 January 2027. And the card price moves too: at half the card price, the card’s share of every local figure above halves.
Bottom line#
A million output tokens costs 0.45 EUR at the small API tier and 36.32 EUR at the frontier, as of November 2026. On an RTX 5090 at today’s price it costs 0.21 EUR with 32 busy streams, 1.15 EUR with one busy stream, and 109 EUR when a single person uses it normally. Local wins on price only at high, steady volume, and it gives you a smaller model for that price. If your reason to run locally is data that must not leave the building, the price is what you pay for that. If your reason is saving money, do the calculation above with your own volume first.