Glossary60 terms
Every "In plain words" box from the posts in one place. The post link jumps straight to the explanation in context. Understanding AI →
A
- AI agent
A large language model put in a loop and given tools: small programs it can run, like "list the files in a folder" or "search the web". The model decides which tool to use, something runs it, the result comes back, and the model decides what to do next. Round and round until the job is done.
- Arithmetic intensity
How much maths you do for each byte you load from memory. High intensity means you reuse loaded data a lot; low intensity means you barely touch it before needing more.
B
- Backpropagation
Training by sending one error signal backward through the whole network, layer by layer. It works extraordinarily well, and conventionally uses a train/run split; its biologically implausible part is weight transport (the backward pass needs an exact mirror of every forward weight), which is nothing a biological brain could plausibly do.
- Base model
The result of pretraining. It has absorbed how language works and a great deal of world knowledge, but it only knows how to continue text, not how to follow an instruction or hold a conversation.
- Batching
Running several requests through the model together in one pass, so they share the cost of loading the weights from memory. Because decode is starved for work, not for maths, adding more sequences to the batch buys you near-linear throughput until you finally run out of memory or compute.
- Blast radius
How much damage an action does if it goes wrong, and how hard it is to undo. Reading a file: tiny, instant to recover. Dropping a database table: enormous, maybe unrecoverable. Autonomy should be rationed by blast radius, nothing else.
C
- Context window
How much text the model can hold in mind at once, measured in tokens. A 128k context is roughly an average novel's worth. Go past it and the model starts forgetting the beginning.
How to read a model's name and specsAlso in: How an agent uses tools and memory, What the model remembers: the context window
- Copyright
Copyright protects a human's creative expression, not ideas, facts or effort. It arises automatically, no registration needed in the EU. Three separate questions follow for AI: is the output protected at all, does the output copy someone else's protected work, and was the training on other people's work allowed.
D
- Differential testing
Instead of asserting what the output should be, you run two versions and compare them to each other. You never have to write down the expected answer, which is why it works on a codebase nobody wrote assertions for. The catch: it only tells you what changed, never whether the old behaviour was right in the first place.
- Diffusion model
An image or video generator that learns to turn random noise into a clean picture through many small denoising steps. The text prompt steers what it denoises towards. This is the engine behind tools like Stable Diffusion, Midjourney, and most AI video.
AI that isn't an LLMAlso in: How an image generator creates a picture from noise
- Distillation
Training a small model (the student) to imitate a large, already-trained model (the teacher). Instead of only learning from text written by people, the student learns from the teacher's predictions about that text. The result is a model that is cheaper to run but keeps a large share of the teacher's ability.
- Draft model
A small, fast model (often a tiny version of the same family) whose only job is to propose candidate tokens. It's allowed to be wrong; the big model is the judge. Cheap guesses that the expensive model grades in bulk.
E
- Embedding
A list of numbers representing the meaning of some input. "Dog" and "puppy" get nearby numbers; "dog" and "tax law" get distant ones. Comparing embeddings is how a computer measures "these two things are about the same topic".
AI that isn't an LLMAlso in: Embeddings: how text becomes a list of numbers, Pointing an LLM at your own documents (RAG)
F
- Feedback Alignment
Instead of the exact transposed weights backprop needs, project the output error back through a fixed random matrix. Astonishingly, the network learns to align with it. Each synapse still updates with a purely local rule; the random feedback just tells each hidden neuron which way is "better". Deep credit assignment, no weight transport.
- Fine-tuning (instruction tuning)
Continuing to train a base model on a smaller, focused dataset to teach it a behaviour or domain. Cheap compared to pretraining, because the heavy lifting is already done; you're adjusting the dials, not setting them from scratch.
G
- GDPR
GDPR (Regulation (EU) 2016/679) protects people's data: whenever a name, an email or a customer ticket goes into a prompt, you are processing personal data and the GDPR applies. The AI Act (Regulation (EU) 2024/1689) regulates AI systems by their use: a chatbot that drafts emails and one that ranks job applicants run on the same model and land in very different rule sets.
- GPU (graphics card)
A chip built to do a huge number of simple calculations at the same time. Originally for drawing game graphics, now the main hardware for AI.
What a GPU is, and why AI needs oneAlso in: How imp turns a model file into words
- Gradient descent
The strategy of always stepping in the downhill direction of the loss. "Gradient" is just the mathematical word for "which way is down, and how steep". Do it enough times and you drift towards the bottom of a valley, where the model is reliably right.
H
- Hallucination
When a model states something false with total confidence. It happens because the model is predicting plausible text, not true text. It has no fact-checker inside.
What an LLM is, and how it produces wordsAlso in: Why models make things up
I
- Independent oracle
A second, simpler implementation of the same calculation, derived from the specification rather than copied from the code under test. If the fast version and the oracle disagree, something is wrong. The catch: it only works if the oracle is genuinely independent.
K
- Knowledge cutoff
The point in time where a model's training data ends. Anything that happened or was written after it isn't in the model, unless someone hands it over in the conversation.
- KV cache
The model's short-term memory for the current conversation. It saves the model from re-reading everything from scratch every time it writes a new word.
L
- LLM (large language model)
A computer program that has read an enormous amount of text and learned its patterns well enough to continue any piece of writing in a believable way. Large = huge (both the text it learned from and its own size), language = it works on text, model = a system that captures patterns.
- Local (or "on-device") AI
Running the model on hardware you own, so your text is processed on your machine instead of being sent to a remote server.
- LoRA (low-rank adaptation)
The original weights stay frozen. Next to each big weight matrix you train two thin matrices whose product is a small correction to it. Only those thin matrices learn. The result is a small adapter file that sits on top of the unchanged base model, like a clip-on lens.
- Loss
One number that says how wrong the model just was. The entire goal of training is to make this number small, averaged over the whole dataset.
M
- MCP (Model Context Protocol)
An open standard for connecting AI applications to tools and data. A system is wrapped once as an MCP server; any assistant that speaks MCP can then use it. It does for AI tools roughly what USB did for peripherals: one plug shape, so N assistants and M systems need N + M adapters instead of N x M.
- MTP head (multi-token prediction)
One extra transformer layer trained together with the model. It takes the model's last hidden state plus the token just produced and predicts the token after that. It is cheap because it is a single layer, and good because it sees the big model's own internal state rather than only the text.
- Multimodal model
A model that handles more than one kind of input, text plus images, sometimes audio or video. "Modality" just means a type of data. The core idea is converting every modality into the one internal language the model speaks: lists of numbers.
N
- Neural network
A web of simple units ("neurons"), loosely inspired by the brain, arranged in layers. Each unit is wired to the next layer by connections, and every connection carries a number called a weight.
O
- Open weights
The trained model file is published and you may download and run it. That's all the term promises. It says nothing about how the model was trained, on what data, or what you're allowed to do with it. Those answers are in the license.
P
- Parameters (the "30B")
The count of numbers inside the model, here 30 billion. Bigger usually means more capable, and always means it needs more memory. This is the single number people quote most.
- Price per million tokens
The API meter. Every request is counted in tokens, input (your prompt, history, documents) and output (the answer) separately, and each has its own price per million. A 500-token answer at 10 USD per million output tokens costs half a cent.
- Prompt injection
An attack where malicious instructions are smuggled into content the model processes (a web page, an email, a PDF, a code comment), so that when your AI reads that content, it obeys the attacker instead of you. The data becomes the command.
- Provider
Provider is whoever develops an AI system or model and puts it on the market under their own name. Deployer is whoever uses one under their own authority at work. A company that rolls out a bought chatbot to its staff is a deployer. Put your own name on a high-risk system, or change a system's purpose so that it becomes high-risk, and you become its provider (Art. 25), with far more duties.
Q
- Quantisation
Storing each of a model's numbers using fewer bits, so the whole thing takes up less space (and runs faster), in exchange for a tiny loss of precision.
R
- R-STDP (reward-modulated STDP)
The same local timing rule, but the change only sticks when a reward signal says "that was good". Because the reward usually arrives well after the spike pair, R-STDP relies on an eligibility trace: a slowly decaying tag on the synapse that bridges that delay, so the later reward can act on the right change. It's how axo turns "fire together" into "fire together when it pays off", which is what learning to act requires.
- RAG (retrieval-augmented generation)
Instead of hoping the model memorised your documents, you fetch the relevant snippets from your own data at question time and paste them into the prompt, then ask the model to answer from that text. Retrieve first, then generate. The model supplies the language; your documents supply the facts.
- Reasoning model
A model trained to write out its working before answering. The working-out is ordinary generated text, produced one token at a time exactly like the answer. It is usually hidden, or shown collapsed, and you are charged for it.
- Refuted
Imp's word for an idea that was tested on a named model and card and found not to be true. The verdict goes into a ledger, the data and usually the diff go into the repo, so the idea can be revived on purpose and not rediscovered by accident.
- RLHF (reinforcement learning from human feedback)
Instead of one "correct" answer to copy, the model gets a reward signal for producing responses people rate as better, and is nudged to earn more reward. It's how raw helpfulness gets shaped into a consistent, well-behaved tone.
- Runtime
The program that loads a model file and does the calculations. The model is the knowledge, a big file of numbers. The runtime is what turns those numbers into answers. Ollama is a runtime, and so is imp.
S
- Sampling
Instead of always taking the highest-scoring word, the model draws one at random, but weighted by the scores. A 41% word gets picked roughly 41% of the time, an 8% word roughly 8% of the time. Likely words usually win; unlikely ones occasionally sneak through. That weighted dice-roll is why you get variety.
- Serialisation
Turning objects in memory into a file and back. Python's built-in way to do it is called pickle, and a pickle file is not just data: it is a small program that the loader executes to rebuild the objects. Whoever wrote the file decides what that program does. Loading a pickle from a stranger is running a stranger's code.
- Shadow AI
AI tools used for work without the company knowing: a free personal chatbot account, a browser extension, a transcription app on a private phone. No contract, no control over whether your input is used for training, no log, no way to delete anything when someone leaves.
- Single source of truth
A fact lives in exactly one place, and everything else derives from it. Here, make build reads the tag out of this file and injects it into the Docker build, so along the make path the container can't drift from the build config. The catch: the Dockerfile keeps its own matching default (a second copy of the version) so a bare docker build still works. The single-source guarantee holds for the make path; a bare docker build that ignores this file can still drift, so the two copies have to be bumped together.
- Spectrogram
A picture of sound. Time runs left to right, pitch runs bottom to top, and brightness shows how loud each pitch is at that moment. A vowel shows up as a few bright horizontal stripes, a hissing "s" as a smear high up, silence as a dark gap. Speech models use a log-mel spectrogram: the pitch axis is squeezed the way human hearing is (we tell low notes apart better than high ones) and loudness is on a log scale, the way we perceive it.
- STDP (spike-timing-dependent plasticity)
The classic "neurons that fire together wire together", made precise by timing. If neuron A fires just before B, their connection strengthens; just after, it weakens. Purely local, and entirely unsupervised.
- Subnormal
The tiny numbers a float format can still store below its normal range, by giving up precision bits instead of exponent range. They let a value fade out gradually instead of jumping to zero, but each step down costs accuracy.
T
- Temperature
A dial, usually 0 to about 2, that controls how much randomness goes into the pick. Low sharpens the odds toward the top choice (safe, focused, repetitive). High flattens them so unlikely words get a real chance (varied, creative, riskier). At 0 it's effectively greedy: always the top word.
- Token
The model doesn't quite work in whole words. It works in tokens, which are short chunks of text, often a word, sometimes part of one. For this article you can just think "word."
What an LLM is, and how it produces wordsAlso in: Why an LLM miscounts the r's in 'strawberry', How imp turns a model file into words
- Tokens per second (tok/s)
How fast it writes. For a sense of scale, a person reads around 5 to 8 words per second, so anything above that already arrives faster than you can read. 300 tok/s feels instant.
- Tool
A small, ordinary program the agent is allowed to run, like read a file, run a search, send an email, query a database. The model can't do any of these itself. It can only ask for them by name.
- Tool calling (also "function calling")
The model is shown a list of tools and their expected arguments, and replies with a structured request naming one. The model only ever produces text; tool calling is the agreed format that lets a plain program read that text and run the right thing.
- Training
Before you ever use it, the model is shown mountains of text and made to play "guess the next word" billions of times. Each time it guesses wrong, every connection weight gets nudged a little in the direction that would have helped. Do that long enough and those weights quietly soak up grammar, facts, and styles.
- Transmission
Thin things aren't opaque. A blade of grass with the sun behind it doesn't go into shadow, it lights up from within, because the light passes through the leaf instead of bouncing off it. It's the reason a meadow at sunset looks lit from inside and the same meadow at noon looks flat.
V
- VRAM
The graphics card's own built-in memory. The model has to fit in here to run fast. A model that doesn't fit either runs very slowly or won't load at all.
W
- Watt (W)
Watt (W) is how fast a device draws power right now. Watt-hour (Wh) is the energy used: 1 W for one hour. Heating one litre of water from 20 to 100 °C takes about 93 Wh in the water alone. Against that, 0.24 Wh is roughly one four-hundredth of a kettle.
- Weights
The billions of numbers inside the model, one on every connection in its neural network. They start as random noise. Training is the process of slowly setting them to useful values. The trained weights are the model.
How a model learns: training, in plain wordsAlso in: How imp turns a model file into words
Z
- Zero data retention (ZDR)
A contract option where the provider does not store your prompts and answers after the request is processed, not even in the abuse monitoring log. It is not a switch you flip yourself: at OpenAI, Anthropic and Microsoft it needs the provider's approval or a separate agreement, and some features that keep state on the server are excluded.