# How an agent uses tools and memory
> The loop, one level down: how the model actually requests a tool, why its whole memory is just the growing transcript, and why long multi-step tasks fall apart.
Source: https://rfriedmann.de/blog/how-an-agent-uses-tools-and-memory/
Published: 2026-06-17 · Track: learn · Level: Advanced
[An agent is a language model in a loop with tools](/blog/what-is-an-ai-agent/):
think, act, observe, repeat. That is the shape. This post opens up the two parts
that decide whether an agent is genuinely useful or just an impressive demo: how a
tool call really works, and what the agent uses for memory.
## How the model asks for a tool
At the start of a task, the runner hands the model a menu: here are the tools you
have, what each does, and the arguments each one takes. In practice that's a
machine-readable list, one entry per tool:
```json
{
"name": "read_file",
"description": "Return the text contents of a file.",
"arguments": { "path": "string, the file to read" }
}
```
When the model wants to use one, it doesn't write a sentence about it. It emits a
structured request the runner can parse exactly:
```json
{ "tool": "read_file", "arguments": { "path": "downloads/report.pdf" } }
```
The runner validates the request (is this a real tool? are the arguments the right
shape?), runs it, and appends the result back into the conversation as a new entry,
usually tagged with a `tool` role so the model can tell its own words apart from
what came back from the world. Then the model takes another turn. That handshake,
model proposes, runner executes, result returns, is every step of the loop.
## The memory is just the transcript
Here is the part that surprises people: an agent has no separate memory. What it
"remembers" mid-task is simply the text so far, the goal plus every think, act, and
observe, all concatenated and fed back in on every single turn.
Picture the window as a fixed-width strip that the running task slowly fills from the left:
The context window fills as the task runs
[diagram omitted — see the page for the chart]The transcript only ever grows, never shrinks, so a long enough task eventually runs out of room.
So "memory" and "the prompt" are the same thing. Each turn, the runner rebuilds the
whole transcript and the model re-reads it from scratch. (The engine caches the
heavy maths so re-reading is cheap, which is the [KV cache](/blog/kv-cache-and-long-context/)
story, but conceptually the model starts every turn by reading everything again.)
This is elegant and it is also the central limitation. The transcript only grows: a
twenty-step task carries twenty steps of tool output along with it. Two things go
wrong as it fills up.
A short task
Goal and a few tool results fit easily
The model sees the whole history every turn
Each decision is made with full information
A long task
Transcript grows past the context window
Early steps must be dropped or summarised
The model starts forgetting its own goal
## Making memory last longer
Because the window is finite, real agents don't just let the transcript balloon.
The common fixes:
Summarise the old turns
When the history gets long, compress the early steps into a short recap and keep only that. The agent trades fine detail for room to keep going.
Keep a scratchpad
Have the agent write its plan and findings to a small running note, a file or a to-do list, so the important state lives in a few lines instead of buried in pages of tool output.
Store outside, fetch on demand
Put big things (documents, past results) in external storage and give the agent a search tool. Now it pulls in only the few relevant pieces when it needs them, instead of carrying everything at once.
That last one is why so many agents come with a search or retrieval tool: it turns
"remember everything" into "look it up when needed", which scales far better.
## Why long plans fall apart
Knowing all this, the failure mode of ambitious agents stops being mysterious. Each
step is a fresh guess, conditioned on a transcript that's getting longer and
noisier with every turn. Three things compound:
- **Errors stack.** A small wrong turn at step 3 becomes the context the model
trusts at step 4, and the mistake calcifies into the plan.
- **The goal fades.** As tool output piles up, the original instruction is a smaller
and smaller fraction of what the model is reading. Late in a long run it can
genuinely lose the thread.
- **Bad tools poison the well.** A tool that dumps thousands of lines of output eats
the budget and drowns the signal. Good agent tools are quiet: they return the
answer, not the haystack.
None of this means agents do not work. It means the ones that work are kept on a
tight rein: small, well-scoped tasks, tools that return little, and a check at the end
rather than blind faith in a twenty-step chain. The skill is not writing a clever
prompt, it is keeping the loop short enough that the guessing stays grounded.
Which leads straight to the question that matters once an agent can touch real
systems: [when do you let it act on its own, and when do you keep a hand on the
wheel](/blog/letting-an-agent-loose/)?