The plain version
A language model can only produce text. It can't open a file, check the weather or add up a spreadsheet. Tool calling is the agreement that gets around this: the app tells the model which tools exist, and when one would help, the model writes a small, structured request instead of an answer, such as get_weather(city="Calgary"). The app runs the real function, hands back the result, and the model carries on.
The model never runs anything itself; it only asks. Your app decides what actually happens. That's both the safety story and the reason the app matters as much as the model.
An everyday analogy. A manager with an assistant. The manager (the model) can't use the filing cabinet, but they can write "please fetch the 2025 Henderson contract" on a sticky note. The assistant (your app) fetches it, and the manager reads it and replies. Give the assistant a list of what they're allowed to fetch, and you control the risk.
What MCP adds. Before MCP, every app wired up every tool its own way. The Model Context Protocol is an open standard for that wiring. Its specification describes hosts (apps such as an editor or a chat UI), the clients inside them, and servers that offer three kinds of things:
- Tools: functions the model can call.
- Resources: data it can read, such as files or database rows.
- Prompts: ready-made templates.
Messages use JSON-RPC 2.0. The spec says it takes inspiration from the Language Server Protocol, which did the same job for programming-language support in editors. Write a server once and any MCP host can use it: Open WebUI, coding agents, desktop assistants, and llama.cpp's own web UI.
Will a local model do it well? That depends on training and on the chat template, which formats the tool list in the exact way the model learned. Recent Qwen3, gpt-oss, Llama 3.x and Mistral models are trained for it. Older or heavily quantized models tend to invent tool names or produce broken JSON. llama.cpp supports native formats for many model families and falls back to a generic format otherwise.
One round trip (OpenAI-style, as llama-server and Ollama accept)
→ request: messages + tools: [{type: "function", function: {name, description, parameters: <JSON Schema>}}] ← response: message.tool_calls: [{function: {name: "get_weather", arguments: {"city": "Calgary"}}}] → request: messages + that assistant message + {role: "tool", content: "<result>"} ← response: final answerllama.cpp
llama-server -hf ggml-org/gemma-3-4b-it-GGUF:Q4_K_M --jinja --port 8080--jinja(on by default in current builds) renders the model's own template, which is what enables native tool calls. If the logs showChat format: Generic, the template wasn't recognised; the docs suggest overriding it with--chat-template-file. Parallel calls are off by default; send"parallel_tool_calls": trueto enable them on models that support it.MCP transports
MCP defines a local transport (the host launches the server as a subprocess and talks over stdio) and a remote one (Streamable HTTP). Remote servers can require OAuth-based authorisation under the spec.
Pitfalls
- Treat tool output as untrusted. A web page or document returned by a tool can contain instructions aimed at the model ("prompt injection"). Don't give a model with web or mail access the power to send, delete or pay without a human confirmation step.
- Least privilege. Point file-system servers at one folder, not your home directory.
- Quantization hurts JSON first. If calls break at Q3, try Q4_K_M or higher before blaming the app.