# Your first local model, step by step > From nothing to a model answering on your own computer: which size to pick, four commands, how to see what it's using, and how to remove it again. Source: https://rfriedmann.de/blog/your-first-local-model/ Published: 2026-11-07 · Track: learn · Level: Beginner So far this course has been about how things work. This lesson is different: at the end of it, a language model runs on your own computer, with no account and no subscription. You don't need a special graphics card to try it. A small model runs on an ordinary laptop, just more slowly. I use a tool called Ollama here, because it handles the setup details and runs on Windows, macOS and Linux. My own engine, [imp](https://github.com/kekzl/imp), isn't the right start: it's built for one specific card. If you'd rather click than type, [LM Studio](https://lmstudio.ai/) does the same job with a window and buttons. ## Step 1: pick a size that fits The one question that matters is how much memory you have. On a computer with a suitable graphics card, that's the card's VRAM. Without one, it's your normal RAM, and the model runs on the processor. [How a model fits on a card](/blog/how-a-big-model-fits-on-one-card/) explains why size is the deciding factor. | Your memory | try | download | |---|---|---:| | 4 GB | `qwen3:1.7b` | 1.4 GB | | 8 GB | `qwen3:4b` | 2.5 GB | | 16 GB | `qwen3:8b` | 5.2 GB | | 24 GB | `qwen3:14b` | 9.3 GB | | 32 GB or more | `qwen3:30b` | 18.6 GB | These are cautious starting points, and the sizes are from Ollama's library on 7 November 2026. Qwen3 is only an example, and the library changes all the time. With what you know from [reading model specs](/blog/reading-model-specs/), you can judge any other entry the same way. If in doubt, start one row lower. And if the model is meant for work, check its license first: [what "open" means for a model](/blog/open-weights-and-licenses/) explains what to look for. ## Step 2: install it On Windows and macOS, download the installer from [ollama.com/download](https://ollama.com/download) and run it. On Linux there's a script: ```bash curl -fsSL https://ollama.com/install.sh | sh ``` That runs a script from the internet with your rights, so read it first if that bothers you. If you already use Docker, you can skip the install: ```bash docker run -d --name ollama -p 127.0.0.1:11434:11434 ollama/ollama ``` In that case, put `docker exec -it ollama` in front of the `ollama` commands below. ## Step 3: start a conversation Open a terminal and type this, with the name you picked in step 1: ```bash ollama run qwen3:8b ``` The first time, it downloads the file, so this step takes as long as your connection needs. After that you get a prompt. Type something and press Enter. Qwen3 is a model that [thinks before it answers](/blog/why-models-think-before-answering/), so you may see it reasoning first. Type `/bye` to leave. The model now runs from a file on your own disk and keeps working without a network connection. ## Step 4: look at what it's using While a model is loaded, open a second terminal and ask what's running: ```bash ollama ps ``` This is what I got for the smallest Qwen3 on a machine with no GPU in use, shortened a little: ```text NAME SIZE PROCESSOR CONTEXT RUNNER qwen3:0.6b 1.0 GB 100% CPU 4096 llamacpp ``` Three things are worth reading here: - **SIZE** is memory in use, and it's bigger than the download. That file was 522 MB. The rest is working memory, much of it for the [context window](/blog/the-context-window/). - **PROCESSOR** says where it runs. `100% GPU` is what you want on a graphics card. A split like `40%/60% CPU/GPU` means the model didn't fit, and it will be much slower. Pick a smaller one. - **CONTEXT** is how much text it can hold at once, here 4096 tokens. That's less than the model can do. It's a default that saves memory. ## Step 5: use it from other programs Ollama also answers on a local address, in the same format as the big cloud APIs. Many tools can talk to it if you give them this address, and you can try it by hand: ```bash curl http://localhost:11434/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model": "qwen3:8b", "messages": [{"role": "user", "content": "Reply with the word ok."}]}' ``` One thing to watch here. With `"max_tokens": 40` the answer came back empty: the model spent all 40 tokens on thinking and never got to the reply. If you set a limit, leave room. ## What to expect - **It's less capable than the big services.** A model that fits at home is much smaller, and it [makes things up](/blog/why-models-make-things-up/) more often. Try it on your own tasks before you trust it. - **It knows nothing new.** It has no web access here, and its [reading stopped at a cutoff](/blog/why-models-dont-know-yesterday/). - **Your text stays with you,** as long as you run a normal local model. Ollama's library also lists cloud models, which run on someone else's server, so look at what you pick. - **It can be slow.** Without a graphics card, a few words per second is normal for a mid-sized model. [Why a GPU helps](/blog/what-is-a-gpu/) is its own lesson. ## Cleaning up Models are big files, and removing them is easy: ```bash ollama list ollama rm qwen3:8b ``` I ran these commands with Ollama 0.40.15 in a Linux container on the processor, using `qwen3:0.6b`. The larger models in the table are sizes from the library, not something I tested for this lesson. You now have a working local model. From here, [why you'd run a model yourself](/blog/why-run-ai-locally/) is worth a read, now that you've got one to compare against.