Archive
Explainers, use cases, how-to guides and issues. Newest first, with review and publication dates labelled.
28 of 28 articles · Newest first
How-to · Setup
Run your first local model with Ollama and Open WebUI
Install Ollama, download a model that fits your machine, chat with it in the terminal, then put a ChatGPT-style web interface on top with Open WebUI.
Reviewed
Sourced, not yet testedHow-to · Setup
Build llama.cpp and run a model with GPU offload
Compile llama.cpp from source with CUDA, download a GGUF straight from Hugging Face, and control how many layers go to the graphics card so models bigger than your VRAM still run.
Reviewed
Partly testedHow-to · Setup
Connect a local model to a coding agent (Aider + Ollama)
Pair-program in your terminal with a model running on your own machine. Aider edits files in your git repository and commits each change; Ollama serves the model.
Reviewed
Sourced, not yet testedHow-to · Setup
Set up private document search (RAG) with Ollama and Open WebUI
Ask questions about your own PDFs and notes with a model that never leaves your machine. Uses a local embedding model, an Open WebUI knowledge base, and a chat model with enough context to read what it finds.
Reviewed
Sourced, not yet testedHow-to · Setup
Run a tiny local model on your Mac CPU
Compile one small program, download a 61 MB model, and generate a story locally. A setup check with no GPU, account or API key.
Reviewed
Tested by usExplainer · Models
What do context length, RoPE and YaRN mean, and how much context do you really get?
Why "128K context" on a model card doesn't mean 128K in your app, how RoPE scaling stretches a model past its training, and what each extra token costs in memory and speed.
Reviewed
Explainer · Models
What is a mixture-of-experts model, and why does "30B-A3B" run so fast?
Why a model can be 117 billion parameters and still type quickly on a mini PC, what "active parameters" means for your memory and your speed, and when MoE is the wrong choice.
Reviewed
Explainer · Formats
GGUF, EXL2/EXL3, AWQ or GPTQ — which quantized file should you download?
The four file formats you'll see on every model page, which runtime each one belongs to, and a one-line rule for picking the right file for your hardware.
Reviewed
Explainer · Apps
What are RAG and embeddings, and how does a local model "read" your documents?
How retrieval-augmented generation lets a small local model answer from your own files without retraining, what an embedding actually is, and the four settings that decide whether it works.
Reviewed
Explainer · Speed
What is speculative decoding, and is it a free speed-up?
How a small "draft" model can make a big one type 2–3× faster with identical output, why the speed-up depends on what you're writing, and how to switch it on in llama.cpp and vLLM.
Reviewed
Explainer · Apps
What are tool calling and MCP, and can a local model use them?
How a model "uses" a calculator, a search engine or your file system without ever running code itself, what the Model Context Protocol standardises, and what to check before you trust a local model with tools.
Reviewed