# What are tool calling and MCP, and can a local model use them?

> With tool calling, the model replies with a structured request (a function name plus JSON arguments) instead of prose. Your app runs the function and returns the result. MCP is an open protocol, built on JSON-RPC 2.0, that lets any compatible app plug into servers offering tools. Local models handle it well if they were trained for it.

Canonical: https://thelocalbench.com/learn/tool-calling-and-mcp · Data as of 2026-10-12 · The Local Bench (thelocalbench.com), data as of 2026-10-12, CC BY 4.0

Flags: [verified] checked against primary source · [sourced] cited · [estimate] formula · [sample] placeholder

Reviewed 2026-10-13 · REV 2026.10

## The plain version

A language model can only produce text. It can't open a file, check the weather or add up a spreadsheet. **Tool calling** is the agreement that gets around this: the app tells the model which tools exist, and when one would help, the model writes a small, structured request instead of an answer, such as `get_weather(city="Calgary")`. The app runs the real function, hands back the result, and the model carries on.

The model never runs anything itself; it only asks. Your app decides what actually happens. That's both the safety story and the reason the app matters as much as the model.

**An everyday analogy.** A manager with an assistant. The manager (the model) can't use the filing cabinet, but they can write "please fetch the 2025 Henderson contract" on a sticky note. The assistant (your app) fetches it, and the manager reads it and replies. Give the assistant a list of what they're allowed to fetch, and you control the risk.

**What MCP adds.** Before MCP, every app wired up every tool its own way. The **Model Context Protocol** is an open standard for that wiring. Its specification describes *hosts* (apps such as an editor or a chat UI), the *clients* inside them, and *servers* that offer three kinds of things:

- **Tools**: functions the model can call.
- **Resources**: data it can read, such as files or database rows.
- **Prompts**: ready-made templates.

Messages use JSON-RPC 2.0. The spec says it takes inspiration from the Language Server Protocol, which did the same job for programming-language support in editors. Write a server once and any MCP host can use it: Open WebUI, coding agents, desktop assistants, and llama.cpp's own web UI.

> **Why it matters for your rig**
>

Tool definitions and results eat **context**. A dozen MCP tools with full descriptions can take several thousand tokens before you've typed anything, and agent loops add every result on top. Ollama's documentation recommends at least 64,000 tokens for agents and coding tools, but its default on GPUs under 24 GiB is 4K. Raise it, and budget the [KV cache](/learn/kv-cache) memory to match. MoE models such as Qwen3-30B-A3B or gpt-oss-20b are a good fit, because agent loops are long and decode speed matters.



**Will a local model do it well?** That depends on training and on the **chat template**, which formats the tool list in the exact way the model learned. Recent Qwen3, gpt-oss, Llama 3.x and Mistral models are trained for it. Older or heavily quantized models tend to invent tool names or produce broken JSON. llama.cpp supports native formats for many model families and falls back to a generic format otherwise.

## The technical layer

### One round trip (OpenAI-style, as llama-server and Ollama accept)

```
→ request:  messages + tools: [{type: "function", function: {name, description, parameters: <JSON Schema>}}]
← response: message.tool_calls: [{function: {name: "get_weather", arguments: {"city": "Calgary"}}}]
→ request:  messages + that assistant message + {role: "tool", content: "<result>"}
← response: final answer
```

### llama.cpp

```
llama-server -hf ggml-org/gemma-3-4b-it-GGUF:Q4_K_M --jinja --port 8080
```

`--jinja` (on by default in current builds) renders the model's own template, which is what enables native tool calls. If the logs show `Chat format: Generic`, the template wasn't recognised; the docs suggest overriding it with `--chat-template-file`. Parallel calls are off by default; send `"parallel_tool_calls": true` to enable them on models that support it.

### MCP transports

MCP defines a local transport (the host launches the server as a subprocess and talks over **stdio**) and a remote one (**Streamable HTTP**). Remote servers can require OAuth-based authorisation under the spec.

### Pitfalls

- **Treat tool output as untrusted.** A web page or document returned by a tool can contain instructions aimed at the model ("prompt injection"). Don't give a model with web or mail access the power to send, delete or pay without a human confirmation step.
- **Least privilege.** Point file-system servers at one folder, not your home directory.
- **Quantization hurts JSON first.** If calls break at Q3, try Q4_K_M or higher before blaming the app.

## Sources

- [Model Context Protocol specification, 2025-11-25](https://modelcontextprotocol.io/specification/2025-11-25)
- [Model Context Protocol specification, 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18)
- [llama.cpp docs: function calling](https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md)
- [Ollama blog: tool support](https://ollama.com/blog/tool-support)
- [Ollama docs: context length](https://docs.ollama.com/context-length)

Try it: https://thelocalbench.com/configurator?mode=hardware&hw=rtx-3090-24g-used