# Run your first local model with Ollama and Open WebUI

> Install Ollama, pull Gemma 3, check the local API answers on port 11434, and run Open WebUI in Docker on localhost:3000 so you can chat in a browser. Use local models and disable external connections for local-only inference.

Canonical: https://thelocalbench.com/how-to/first-local-model-ollama-open-webui · The Local Bench (thelocalbench.com), data as of 2026-10-13, CC BY 4.0

- Difficulty: Beginner
- Time: about 30 minutes
- Cost: $0 (free software; uses the computer you already have)
- Hardware: Any 64-bit computer with 8 GB RAM for small models; 16 GB+ RAM or a GPU with 8 GB+ VRAM for 12B-class models
- OS: Linux, macOS, Windows
- Prerequisites: An administrator account to install software; Docker Desktop or Docker Engine (step 4 only); About 10 GB of free disk space
- Parts: Ollama (local model runtime); Open WebUI (browser chat interface, runs in Docker); Gemma 3 (4B or 12B) (first model; pick by memory)
- Status: Sourced, not yet tested. Every command is copied from the official Ollama and Open WebUI documentation (checked 2026-10-13), but we have not yet run this guide end to end on our bench.
- Tested on: not yet tested
- Last tested: never
- Reviewed: 2026-10-13

## Not yet tested by us

- The whole guide on every OS
- The Open WebUI :cuda image with --gpus all

## Step 1: Install Ollama

On **Linux**, run the official install script:

```sh
curl -fsSL https://ollama.com/install.sh | sh
```

On **Windows**, in PowerShell:

```powershell
irm https://ollama.com/install.ps1 | iex
```

On **macOS**, download the app from [ollama.com/download](https://ollama.com/download), drag it to Applications, and open it once.

> **Warning: Read before you pipe to a shell**
>
The Linux and Windows one-liners download and run a script. If you'd rather check it first, open the URL in a browser, or use the manual install instructions on the download page.




## Step 2: Download and chat with a first model

Pick a size that fits your memory. Gemma 3 4B runs on most laptops; 12B wants about 16 GB of RAM or an 8 GB+ graphics card.

```sh
ollama run gemma3:4b
```

The first run downloads the model, then opens a chat prompt. Type a question and press Enter; type `/bye` to quit.

> **Tip: Not sure which size?**
>
Gemma 3 4B and 12B are not yet in our fit catalogue, so the calculator cannot validate these examples. The memory figures above are starting points, not measured fit guarantees.


The [configurator](/configurator) estimates memory fit and speed for models in our [catalogue](/models). For a small setup we have run ourselves, try the [CPU generation demo](/how-to/tiny-model-cpu).



## Step 3: Check the local API is running

Ollama serves an API on port 11434. Other apps, including Open WebUI, talk to it there.

```sh
curl http://localhost:11434/api/chat -d '{"model": "gemma3:4b", "messages": [{"role": "user", "content": "Say hello in five words."}], "stream": false}'
```

You should get back a JSON object with a `message` containing the reply.



## Step 4: Run Open WebUI in Docker

With Docker running, start Open WebUI and point it at Ollama on your machine:

```sh
docker run -d -p 127.0.0.1:3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```

Open [http://localhost:3000](http://localhost:3000). The first account you create becomes the administrator.

> **Tip: NVIDIA GPU**
>
The Open WebUI docs also offer a `:cuda` image, started with `--gpus all`, which speeds up its built-in features such as document embedding. Ollama itself already uses your GPU without it.




## Step 5: Verify it works

In Open WebUI, choose **gemma3:4b** from the model menu at the top and ask a question. Then confirm it ran locally:

```sh
ollama ps
```

```text
# Expected output
NAME         ID    SIZE    PROCESSOR    CONTEXT    UNTIL
gemma3:4b    …     …       100% GPU     4096       4 minutes from now
```

The exact columns vary by version. `PROCESSOR` shows whether the model is on your GPU, CPU or split between them.





## Troubleshooting

### Open WebUI shows no models

The container can't reach Ollama. Check that `curl http://localhost:11434` replies "Ollama is running" on the host, and that you included `--add-host=host.docker.internal:host-gateway`. On Linux, Ollama listens on 127.0.0.1 by default; see the Ollama FAQ for setting `OLLAMA_HOST` if the container still can't connect.



### Replies are very slow

Run `ollama ps`. If `PROCESSOR` shows a CPU/GPU split, the model doesn't fit in VRAM; try a smaller tag, such as `gemma3:1b` or `gemma3:4b`.



### The model forgets earlier parts of long chats

Ollama sets the context length from your VRAM, and below 24 GiB that's 4K tokens. Raise it in the Ollama app's settings, or start the server with `OLLAMA_CONTEXT_LENGTH=8192 ollama serve`. Longer context uses more memory.



### Port 3000 is already in use

Change the left side of `-p 127.0.0.1:3000:8080`, for example `-p 127.0.0.1:3001:8080`, and open localhost:3001 instead.

## Sources

- [Ollama: download and install](https://ollama.com/download)
- [Ollama API reference (port 11434, /api/chat)](https://github.com/ollama/ollama/blob/main/docs/api.md)
- [Ollama library: gemma3 tags](https://ollama.com/library/gemma3)
- [Ollama docs: context length](https://docs.ollama.com/context-length)
- [Open WebUI quick start](https://docs.openwebui.com/getting-started/quick-start/)