Choosing an AI provider: a checklist for IT
What to settle before signing with an AI provider: DPA and EU hosting, SSO and audit logs, model retirement, limits and SLAs, pricing, certificates, an exit plan and a test on your own work.Understanding AIOps22 Nov 2026, 7 minAn AI policy for your company: one page people can follow
Why a ban on AI tools only moves use to private accounts, and what a one-page usage policy must say: tools, data classes, review duty, agents, AI literacy. With a skeleton to copy.Understanding AIOps21 Nov 2026, 10 minWhat happens to your prompts at the provider
Training, retention, abuse logs and EU hosting at OpenAI, Anthropic, Google and Microsoft as of November 2026, and why consumer apps and business tiers follow different rules.Understanding AIOps20 Nov 2026, 7 minLocal or API: what a million tokens really costs
API prices per million tokens against an RTX 5090 at 7 700 EUR and 37.3 ct/kWh: the arithmetic, the break-even volumes, and why a lone user pays the most locally.Understanding AIOps19 Nov 2026, 7 minEmbeddings: how text becomes a list of numbers
An embedding turns text into a list of numbers so that similar meaning lands close together. How that powers search and RAG, where it fails, and what the vectors cost to store.Understanding AIBeginner18 Nov 2026, 7 minHow much electricity does an AI answer use?
Published figures put one chatbot answer at 0.24 to 0.34 Wh. What they include, why they spread so far, and my own RTX 5090: 0.115 Wh per answer alone, 0.024 Wh shared by 32.Understanding AIOps17 Nov 2026, 6 minAI and the law: what GDPR and the AI Act ask of your team
Before staff paste work into a chatbot, two EU laws apply: GDPR for the data in the prompt, the AI Act for what the system is used for. What each asks of an IT team, with the dates as they stand in November 2026.Understanding AIOps8 Nov 2026, 7 minFine-tuning on your own data, with LoRA
Fine-tuning is the wrong tool for facts and the right one for style, format and narrow tasks. When it pays off, how LoRA makes it fit on one GPU, and what data and checks you need.Understanding AIOps8 Nov 2026, 6 minHow an image generator creates a picture from noise
An image generator neither paints nor assembles a collage: the model starts from random noise and removes it step by step, steered by your words. How that works, and why it explains the distorted hands.Understanding AIBeginner8 Nov 2026, 7 minHow a model hears and speaks
Dictation, meeting transcripts, voice assistants and cloned voices all rest on one idea: turn sound into a picture, then into tokens. How speech recognition and speech synthesis work, where they fail, and what to watch for at work.Understanding AIBeginner8 Nov 2026, 7 minWhen a downloaded model is safe to run
A model file looks like a set of numbers, and some formats are exactly that. Others run code the moment you load them. Which formats are which, what has already gone wrong, and the checks I'd make before a model touches a work machine.Understanding AIOps8 Nov 2026, 6 minOpen weights, open source, and model licenses
A model you can download for free is not automatically free to use for anything. What "open" means for a model, the license families you'll meet, and five questions to ask before a model goes to work.Understanding AIBeginner8 Nov 2026, 6 minMCP: one plug for every tool an agent uses
The Model Context Protocol is how tools, files and business systems plug into an AI assistant. What the pieces are, what a call looks like on the wire, and what an IT team should allow before anyone installs a server.Understanding AIAdvanced8 Nov 2026, 7 minCopyright and AI output: who owns what the model writes
A generated text, image or function: who owns it, whether it can infringe someone else's work, and the status of the data the model learned from. The court cases that matter as of November 2026, and practical rules for a team.Understanding AIOps8 Nov 2026, 7 minYour first local model, step by step
From nothing to a model answering on your own computer: which size to pick, four commands, how to see what it's using, and how to remove it again.Understanding AIBeginner7 Nov 2026, 5 minThe kernel that ran zero times
A rewrite made an imp kernel up to 33 % faster, and serving never launches that kernel. Four results from one day that changed no engine code, and the ledger that stops the next audit from paying for them twice.The logExpert6 Nov 2026, 9 minWhy a model doesn't know what happened yesterday
A model can explain quantum physics and still get today's date wrong. What a knowledge cutoff is, why a model is out of date on the day it ships, and how to work around it.Understanding AIBeginner6 Nov 2026, 6 minSpeculative decoding when the chats overlap
Qwen3.8-27B ships its own draft head. On imp it sped up one chat by 35 %, and any second chat or UI side request switched it off. Nine changes later, four overlapping chats decode at about 140 tok/s instead of 84.The logExpert4 Nov 2026, 10 minQuantisation lives in the scales
Three imp fixes that changed no weight bit: a block scale that rounded to zero, an activation scale the checkpoint already shipped, and one FP8 scale shared by K and V. Qwen3-30B-A3B perplexity 11.3565 to 10.9566.The logExpert4 Nov 2026, 7 minAPA: sloppy attention, exact where it counts
A prefill kernel for the RTX 5090 that runs every attention tile in FP4, recomputes the 6 to 10 % that carry the softmax mass in exact FP16, and merges both: 2.50x faster than all-exact at 122880 keys.The logExpert28 Oct 2026, 11 minHow a small model learns from a big one
Many of the small open models you can run at home were taught by a bigger model, not just by the internet. What distillation is, why it works so well, and what it can't hand down.Understanding AIBeginner10 Oct 2026, 7 minHow imp runs Qwen3.8-Flash-Next on a 32 GB card
56 GiB of experts, 32 GB of VRAM: all experts on the host, a GPU-managed cache in VRAM, and decode from 5.8 to 75-80 tokens per second on real text.The logExpert27 Sep 2026, 14 minMaking a model emit valid JSON, every time
Asking nicely gets you valid JSON almost always, and almost is a broken integration. The fix is in the sampler: delete every invalid token before you sample.The logExpert16 Sep 2026, 6 minTesting a model on your own work
Leaderboards measure performance on public benchmarks. Forty real cases from your own backlog, graded consistently, show how a model performs on your work.Understanding AIOps15 Sep 2026, 6 minWhy a model thinks before it answers
Modern models write pages of working-out before answering. That 'thinking' is computation bought with tokens: what it buys, what it costs, when it's wasted.Understanding AIBeginner14 Sep 2026, 6 minThe oracle you build yourself
Dependabot opened five PRs against this site, three of them major. No test suite, so I built the check: render the site with both versions and diff it.The logOps11 Aug 2026, 9 minNo oracle for a meadow
One sentence to Claude Opus 5, 1,922 lines of WebGL, a honeybee in backlight, no number to check. What stands in for a test when the only judge is your eye?The logAdvanced10 Aug 2026, 15 minContinuous batching: how one GPU serves a crowd
A single decode stream wastes most of the GPU. Serve many requests at once, share each pass over the weights, and stop waiting for the whole batch to finish.The logAdvanced24 Jun 2026, 4 minPrompt injection: the security risk in LLM applications
The moment your AI reads anything an attacker can influence, that content can hijack it. There is no complete fix, only containment: an operations briefing.Understanding AIOps24 Jun 2026, 5 minHow a model sees a picture
You can hand a modern AI a photo and ask about it. A language model only understands tokens; this is how vision is added to a model that only knew words.Understanding AIBeginner23 Jun 2026, 4 minHow to write a prompt: clear instructions instead of special phrases
Good prompting means clear instructions to a capable, literal-minded assistant with no memory, derived from how the model works.Understanding AIBeginner23 Jun 2026, 4 minSpeculative decoding: let a small model do the guessing
Decode is bandwidth-bound, so the GPU's maths units sit mostly idle. Speculative decoding spends that compute verifying guessed tokens: same output, faster.The logExpert22 Jun 2026, 5 minWhat the model remembers: the context window
An LLM has no memory between messages: it re-reads your whole conversation every turn, and only so much fits. Meet the context window.Understanding AIBeginner22 Jun 2026, 5 minPointing an LLM at your own documents (RAG)
A common request is 'a ChatGPT that knows our internal docs'. That's RAG: simpler than often presented, and most of the work is in the retrieval half.Understanding AIOps21 Jun 2026, 6 minWhy models make things up
A model hands you a wrong fact, a fake citation or an invented function with total confidence. Where 'hallucinations' come from, and how to work around them.Understanding AIBeginner20 Jun 2026, 5 minWhy you get a different answer every time
Ask a model the same thing twice and you can get two different replies. That is deliberate weighted sampling, not a fault; one setting controls how much the answers vary.Understanding AIBeginner19 Jun 2026, 4 minWhy an LLM miscounts the r's in 'strawberry'
Ask a top model to count the letters in a word and it often gets it wrong. The cause is not a bug: the model never sees letters at all. A tour of tokens.Understanding AIBeginner18 Jun 2026, 5 minaxo: learning without backprop
A from-scratch spiking neural network in C++/CUDA on one RTX 5090 that learns with local rules, no backprop, and grows into a creature that learns to hunt.The logAdvanced17 Jun 2026, 7 minBumping a dependency you can't read
Updating CUTLASS under imp's 4-bit maths. In numerical CUDA a bad bump doesn't crash, it returns wrong numbers. You don't review it, you make it verifiable.The logAdvanced17 Jun 2026, 8 minHow an agent uses tools and memory
The loop, one level down: how the model requests a tool, why its whole memory is just the growing transcript, and why long multi-step tasks fall apart.Understanding AIAdvanced17 Jun 2026, 5 minWhen to let an agent act on its own: an operations view
An agent that can act can also act wrongly. After twenty years of running systems: how much autonomy I give one, and which guardrails are worth it.Understanding AIOps17 Jun 2026, 4 minOnline softmax, and the register that can't move
FlashAttention on the 5090, in depth: how online softmax avoids the giant score matrix, and why register-resident output forces raw mma.sync over WMMA.The logExpert17 Jun 2026, 4 minWhat a consumer RTX 5090 is missing next to a datacenter GPU
The 5090 and the B200 are both Blackwell, but the consumer chip lacks whole capabilities. What is gone, what it costs, and why data centre code won't port.The logAdvanced17 Jun 2026, 6 minDecoding GGUF faster than llama.cpp on a 5090
imp decodes dense GGUF 37 to 72% faster than llama.cpp on a 5090. Not because llama.cpp is naive: imp is built for one chip. And where it loses.The logExpert16 Jun 2026, 4 minWhat an AI agent is
An agent is a language model put in a loop and given tools, so it can carry out actions instead of only describing them.Understanding AIBeginner16 Jun 2026, 4 minAI that isn't an LLM
Language models are one family of AI among several. A tour of the other big families, what each is for, and when an LLM is the wrong tool.Understanding AIBeginner15 Jun 2026, 4 minNVFP4 at the bit level
Every other post says '4-bit' and moves on. Here's NVFP4 down to the four bits and two scales, and the two Blackwell instructions (cvt, mma) that make it fly.The logExpert15 Jun 2026, 3 minPretraining, fine-tuning, and RLHF
A raw trained model can continue text but won't answer you. The three stages that turn it into a helpful assistant, and which one you need.Understanding AIAdvanced14 Jun 2026, 4 minWhen not to use AI: an operations view
When reaching for an LLM is the wrong choice, and a simple heuristic for telling those cases apart.Understanding AIOps14 Jun 2026, 3 minHow a model learns: training, in plain words
Every LLM starts as random noise, shaped by one loop: guess, measure the error, nudge billions of dials. How training works, and why it is expensive.Understanding AIBeginner13 Jun 2026, 4 minWhen the model isn't a transformer: GDN and Mamba2
Not every LLM is built on attention. Gated DeltaNet and Mamba2 swap the score matrix for a recurrence: constant memory, and one stubborn precision question.The logExpert13 Jun 2026, 4 minSelf-hosting an LLM for your team: usually not worth it
Why most teams shouldn't run their own model: a small model's error rate costs more working time than the API fees it saves. Plus where it still makes sense.Understanding AIOps12 Jun 2026, 4 minThree kernels for a chip the ecosystem skipped
FlashAttention-2 and NVFP4 GEMM tuned from scratch for the RTX 5090, and what the profiler taught me when every textbook optimisation was a red herring.The logExpert11 Jun 2026, 8 minLLMs in the terminal: a sysadmin's list
The concrete, everyday ways an LLM earns its place in a sysadmin's workflow, and the handful of rules that keep it from causing damage.Understanding AIOps10 Jun 2026, 4 minHow 97,000 lines of CUDA got written by an AI agent
Both of my imp posts end with 'every line was written by Claude Code'. The question that always follows: how does that actually work, and how do you trust it?The logAdvanced9 Jun 2026, 6 minWhat an LLM can and can't do in your infrastructure
Twenty years of running systems, a couple working with LLMs: where AI helps an ops team, and where it is a liability.Understanding AIOps8 Jun 2026, 5 minServing 30B models at 300 tok/s on a single RTX 5090
Why no existing inference engine fully exploits consumer Blackwell, what NVFP4 changes, and the numbers from building one that does.The logAdvanced2 Jun 2026, 6 minThe KV cache, and what really limits long context
The beginner version calls it short-term memory. In reality the KV cache, not the weights, decides how long your context gets and what runs you out of memory.The logAdvanced19 May 2026, 4 minMixture of Experts: how a 30B model runs like a 3B one
The design behind Qwen3-30B-A3B: split the model into many experts, run only a few per token. Why it suits one GPU, and the engineering costs it brings.Understanding AIAdvanced12 May 2026, 3 minPrefill, decode, and the roofline that explains everything
An inference engine doesn't have one speed but two, and they obey opposite laws. The roofline is the most useful lens for reasoning about LLM performance.The logAdvanced5 May 2026, 4 minHow imp turns a model file into words
A plain-language tour of what happens inside an AI engine: how a model gets loaded, and the steps every message runs through to come back as text.The logBeginner28 Apr 2026, 6 minHow to read a model's name and specs
Model names like Qwen3-30B-A3B-Q4_K_M look cryptic at first. How to decode them, and the handful of specs that matter.Understanding AIBeginner17 Apr 2026, 2 minHow a 30-billion-parameter model fits on one card
Quantisation in plain terms: how storing each number in a model with fewer bits lets a large model fit on a desktop graphics card, and what it costs.Understanding AIBeginner10 Apr 2026, 2 minWhat a GPU is, and why AI needs one
Why running AI means buying a graphics card, what makes a GPU different from a CPU, and why the amount of memory on the card is the most important number.Understanding AIBeginner3 Apr 2026, 2 minWhen running AI on your own machine makes sense
Cloud chatbots are easy to use and hard to beat. The narrower case for running a model yourself, and what you give up to do it.Understanding AIBeginner27 Mar 2026, 5 minWhat an LLM is, and how it produces words
A jargon-free explanation of large language models: what they are, why they're basically a very good autocomplete, and how they write one word at a time.Understanding AIBeginner19 Mar 2026, 5 min