Skip to main contentThe Local Bench

Use case 3 of 13

A local coding copilot

Autocomplete in the editor and an agent that edits files and runs tests, with your code staying on your machine.

A laptop, a compact mechanical keyboard and a cup of coffee on grey paper
Image: generated with Grok Imagine

The real example

Continue with Ollama, plus OpenHands for agent work

Continue is an open-source assistant for VS Code and JetBrains with chat, edit, autocomplete and an agent mode, and its docs include guides for using Ollama and running without internet. For multi-step work, the OpenHands agent edits a repository, runs commands and iterates. Devstral Small 1.1, built by Mistral AI with All Hands AI for this style of agent, reports 53.6% on SWE-bench Verified with the OpenHands scaffold. That is a vendor-reported figure, but a useful reference point for what a 24B model can do on real GitHub issues.

Sources: Continue documentation · OpenHands documentation · Devstral Small 1.1 model card

Why run it locally

Proprietary code, client repositories under NDA, or simply no per-seat fees. Autocomplete also feels faster when the model is next to the editor.

Recommended stack

Three tiers built from our builds. Weights-in-memory is our arithmetic (parameters × bits), checked against each device's usable memory; it excludes KV cache and runtime overhead. Runtime compatibility is not validated.

Budget

Value 16 GB

$1,173Sample

16 GB on GeForce RTX 5060 Ti 16 GB

Weights ≈ 13.6 GB of 15.5 GB usableEstimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

A 7B coder for autocomplete; a 14B model for chat.

Runtime: Continue + Ollama

Mid

Used-3090 24 GB

$1,980Sample

24 GB on GeForce RTX 3090 24 GB (used)

Weights ≈ 18.5 GB of 23.5 GB usableEstimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

The 3B-active MoE coder is quick enough for chat and light agent work. It fills most of the card, so swap in the 7B coder when you want autocomplete.

Runtime: Continue + llama.cpp server

Pro

Dual-GPU 48 GB

$4,044Sample

48 GB on 2× RTX 3090 (used) 48 GB

Weights ≈ 18.9 GB of 47.0 GB usableEstimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

Agentic coding needs long context. 48 GB leaves room for it.

Runtime: OpenHands + vLLM; Continue for autocomplete

Setup outline

  1. Install Ollama or llama.cpp and pull a coder model plus a small autocomplete model.
  2. Install the Continue extension and point its chat, edit and autocomplete roles at the local server.
  3. Index your repository so the assistant can use it as context.
  4. For agent work, run OpenHands in Docker against a local OpenAI-compatible endpoint and a sandboxed workspace.
  5. Keep tests green. Let the agent run them and review every diff before merging.

Limits

  • Local models still trail the best hosted models on hard, multi-file changes.
  • Agent loops burn long contexts. Prefill speed (compute), not just memory bandwidth, decides how responsive they feel.
  • SWE-bench numbers are vendor-reported with each vendor's own scaffold. Compare on your own repo.

Next: Receipt and paperwork scanning

Build prices come from our build datasheets: GPUs and devices are sourced retail or sold prices; other parts are sample prices until sourced, and the flag on each total says which. Model links go to our datasheets, where each parameter count and licence was checked against the Hugging Face API.