# When a downloaded model is safe to run
> A model file looks like a set of numbers, and some formats are exactly that. Others run code the moment you load them. Which formats are which, what has already gone wrong, and the checks I'd make before a model touches a work machine.
Source: https://rfriedmann.de/blog/is-a-downloaded-model-safe/
Published: 2026-11-08 · Track: learn · Level: Ops
When someone on the team wants to try a new model, the conversation is usually about
size, speed and [licence](/blog/open-weights-and-licenses/). Almost nobody asks the
question I'd ask about any other file from the internet: *will this run code on my
machine?* The intuition is that a model is just weights, a few gigabytes of numbers,
and numbers can't hurt you.
For some formats that intuition is right. For others it is wrong, and the
difference is a file extension most people never look at. This lesson is about telling
them apart, and about the cheap habits that make the question mostly go away.
## Where code can run
There are three ways a "model" can do more than hand over numbers.
**Pickle checkpoints.** The classic PyTorch formats (`.bin`, `.pt`, `.pth`, `.ckpt`)
are pickle underneath. A pickle can name any Python function to call while it is being
loaded, so a model file can open a reverse shell before you have typed a single prompt.
This has already happened: in February 2024 JFrog reported around 100 models on Hugging
Face that executed code on load, some of them connecting back to a real IP address.
PyTorch 2.6 made `weights_only=True` the default for `torch.load`, which loads only
tensors and a short list of safe types. Before 2.6 even that flag could be bypassed to
run code (CVE-2025-32434), so use 2.6 or later. That helps, but old code, old versions and
every tutorial that says "just add `weights_only=False` to fix the error" still open
the door.
**`trust_remote_code=True`.** Some models on Hugging Face ship their own Python files
for architectures the `transformers` library doesn't know yet. Setting this flag tells
the library to download and run that code. The documentation says it plainly: only for
repositories you trust and whose code you have read. It is not a format problem at all,
it is code execution by design.
**The loader itself.** Formats like GGUF are pure data, but some C code still has to
parse them, and parsers have bugs. In January 2024 Databricks reported five heap
overflows in the GGUF parsing of the `ggml` library that llama.cpp is built on
(CVE-2024-25664 to CVE-2024-25668), patched in llama.cpp within a week. A few months
later a different hole turned up one layer higher: GGUF files carry a chat template,
and `llama-cpp-python` rendered it with an unsandboxed Jinja2 engine, so a crafted
template could run code (CVE-2024-34359, versions 0.2.30 to 0.2.71, fixed in 0.2.72).
Data formats are safe only as long as the program reading them is current.
Model file formats, from "just data" to "runs code by design"
[diagram omitted — see the page for the chart]
The left two are data formats: the risk is a bug in the program that reads them, fixed by updating it. The right two execute code by construction: the only defence is trusting the author, or not loading the file outside a sandbox.
## Scanners help, they don't decide
Hugging Face scans uploaded pickle files for dangerous imports and flags what it finds.
That catches simple attacks, but not more sophisticated ones: in February 2025
ReversingLabs found two models (they called the technique *nullifAI*) that slipped past
the scanner by packing a PyTorch file with 7z instead of ZIP and deliberately breaking
the pickle stream *after* the payload. The scanner refused to analyse a broken file;
Python's loader still ran the instructions up to the break, payload included.
Hugging Face removed the models within a day and updated the scanner, but the lesson
stands: a green badge means "nothing known was found", not "safe".
## Check who uploaded it
Anyone can create an account and name a repository whatever they like. A copy of a
popular model under an organisation name one letter off, or a "quantised" re-upload by
an account with no history, is the easiest supply-chain attack there is. The checks are
simple and fast:
- **Start from the publisher, not from search.** Follow the link from the vendor's own
site, blog post or model card to the organisation page, and download from there.
- **Prefer safetensors or GGUF.** If the only option is a pickle checkpoint, ask why.
Most current releases ship safetensors.
- **Pin what you tested.** A repository can change after you looked at it. Download by
commit (the `revision` argument in `transformers`, or a fixed tag in your tooling) and
keep the SHA256 that Hugging Face shows for every large file.
- **Read before `trust_remote_code`.** If a model needs it, someone reads the Python
files in that exact revision, or the model waits until the library supports it
natively.
- **Keep the runtime current.** llama.cpp, Ollama, vLLM and similar tools are parsers for
files from the internet. Patch them like you patch a browser.
## Defence in depth, because a check can fail
Even with all of the above, I run new models the way [I'd run any unfamiliar
service](/blog/self-hosting-an-llm-for-your-team/): in a container, as a non-root
user, with the model directory mounted read-only, with no access to credentials, home
directories or the corporate share, and with no outbound network unless it is needed.
Keep the container runtime and the NVIDIA Container Toolkit patched as well: a flaw
there (CVE-2025-23266) let a crafted image take over the host.
A reverse shell that cannot reach anything is a log entry, not an incident. The
[step-by-step setup](/blog/your-first-local-model/) in this course works the same way
inside a container; it just takes one more line.
Risky
Grabbing the top search hit from an unknown account
Loading a .bin or .pt because "it's just weights"
Adding weights_only=False to silence an error
trust_remote_code=True without reading the code
Running the model as your own user, next to your SSH keys
Safe
Download via the publisher's own link
safetensors or GGUF, with a current runtime
Pinned revision and recorded SHA256
Container, read-only weights, no secrets, no network
Licence checked in the same step
## Takeaway
The safe path has become the default path. Big releases ship
safetensors and GGUF, PyTorch refuses arbitrary pickles out of the box, and the known
parser holes are patched. The risk that remains lives in the shortcuts: the unverified
re-upload, the flag added to make an error go away, the runtime nobody updated since
spring. Treat a model file like any other download that ends up executing on your
hardware. It's the same discipline as [prompt injection](/blog/prompt-injection/), one
layer further down: assume something hostile will eventually get loaded, and make sure
it finds nothing worth taking when it does.