When not to use AI: an ops take
The unfashionable half of the conversation. The cases where reaching for an LLM is the wrong choice, and a simple heuristic for telling them apart from the good ones.
Plenty of people will tell you where to use AI. Here is the other half, the part that is quietly more valuable in operations: knowing when not to. Reaching for an LLM where it does not belong does not just waste effort, it adds a confident, plausible failure mode to a system that did not have one before.
The four times to put it down
When you need determinism. Same input, same output, every time. That is the foundation of anything automated you actually trust, and an LLM does not offer it: ask twice, get two phrasings, occasionally two different answers. For a pipeline, a check, a gate, that is a non-starter. A script is boring and identical every run. Boring and identical is what production wants.
When a normal tool already does the job. A regex, a jq filter, a parser, a proper
API. If the problem is well-defined, the deterministic tool is faster, free,
testable, and does not hallucinate. Using an LLM to do a regex’s job is paying a
slot machine to do arithmetic.
When you cannot afford to verify, and cannot afford to be wrong. The model is confidently wrong sometimes, and fluently so. If checking its answer is as much work as doing the task, it saved you nothing. If you skip the check because it sounded right, you have shipped a hidden hazard.
When it has to act unattended. Drafting for a human to approve is the ideal case. Letting the model take an action on production with nobody reading it first is how “it usually works” becomes an incident report.
The trap underneath all four
It is the same one every time: the demo works. The model handles the obvious 80% well, so it looks solved. The danger is the other 20%, where it fails not loudly but plausibly, producing something well-formed and wrong that sails right past a tired reviewer. Deterministic tools fail in boring, visible ways. LLMs fail in fluent, invisible ones, which is exactly the failure mode operations is built to dislike.
A heuristic you can actually use
Before reaching for an LLM, two questions:
Good fit if
- The input is messy or natural language
- A human reads the output before it acts
- Being roughly right, fast, is valuable
Wrong tool if
- You need the same answer every time
- A regex, parser, or API already does it
- Nobody will check it, and wrong is costly
That is the whole filter. Messy input plus a human check equals a great use of an LLM. Determinism, ground truth, or unattended action means you should reach for something else.
Read it top to bottom, and stop at the first yes:
None of this is anti-AI. I build entire systems with these models. It is the oldest lesson in operations, just pointed at a new and very persuasive tool: the experienced choice was never using the fashionable thing everywhere. It was knowing exactly where it does not belong, and having the discipline not to put it there.