Skip to main contentThe Local Bench

Use case 1 of 13

A private voice assistant for the home

Wake word, speech recognition, a small language model and a voice, all on a box in the cupboard. Nothing you say leaves the house.

A small black mini PC next to a fabric smart speaker and a blue desk microphone
Image: generated with Grok Imagine

The real example

Home Assistant Assist, fully local

Home Assistant documents a voice pipeline that runs entirely on your own hardware: a local speech-to-text engine, Home Assistant's own intent handling, and a local text-to-speech voice, connected over the Wyoming protocol. Its guide is candid about the trade-off. Whisper is open-ended but slow on small boards: around 8 seconds per command on a Raspberry Pi 4, under a second on an Intel NUC. The closed-vocabulary Speech-to-Phrase engine answers in under a second even on a Pi 4 but only understands home-control phrases. Piper, the local voice, generates 1.6 seconds of speech per second on a Pi with medium-quality voices. To go beyond fixed commands, the Ollama integration plugs a local language model in as the conversation agent, with optional, experimental control of exposed devices.

Sources: Home Assistant: set up a fully local voice assistant · Home Assistant: Ollama integration · Home Assistant: Wyoming integration

Why run it locally

Microphones in every room are the clearest case for local AI. The audio never leaves your network, and the assistant still works when the internet is down.

Recommended stack

Three tiers built from our builds. Weights-in-memory is our arithmetic (parameters × bits), checked against each device's usable memory; it excludes KV cache and runtime overhead. Runtime compatibility is not validated.

Budget

Silent home server

$1,283Sample

16 GB on GeForce RTX 4060 Ti 16 GB

Known weights ≈ 6.6 GB; some model sizes are unknown. Memory fit cannot be determined.Estimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

Whisper turbo plus an 8B model fit together on a 16 GB card with room for context.

Runtime: Home Assistant OS + Wyoming Whisper and Piper apps; Ollama for the LLM

Mid

Used-3090 24 GB

$1,980Sample

24 GB on GeForce RTX 3090 24 GB (used)

Known weights ≈ 18.6 GB; some model sizes are unknown. Memory fit cannot be determined.Estimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

An MoE model with 3B active parameters keeps replies quick while knowing more.

Runtime: Wyoming-compatible ASR server, Ollama or llama.cpp

Pro

Strix Halo SFF 128 GB

$3,499.99Sourced

128 GB on Ryzen AI Max+ 395 mini PC 128 GB

Known weights ≈ 68.5 GB; some model sizes are unknown. Memory fit cannot be determined.Estimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

128 GB of unified memory holds a 120B MoE model next to speech models. Expect slower first replies than on a GPU.

Runtime: llama.cpp server + Wyoming

Speech-to-text accuracy figures are on the speech-to-text ranking; the TTS category has no common public benchmark yet.

Setup outline

  1. Install Home Assistant OS on the server, or run it in a VM next to your GPU host.
  2. Add the Whisper (or Speech-to-Phrase) and Piper apps; the Wyoming integration discovers both.
  3. Create an assistant under Settings › Voice assistants, picking the local speech-to-text and text-to-speech engines.
  4. Install Ollama on the GPU machine, make it reachable on your LAN, and add the Ollama integration as the conversation agent.
  5. Expose only the devices you want the model to control, then test with a phone app or a voice satellite.

Limits

  • Wake-word satellites and room microphones are extra hardware; a phone works for testing.
  • LLM device control is marked experimental by Home Assistant. Keep exposed entities to a minimum.
  • English gets the best speech accuracy. Check the leaderboard and voice quality for your language before buying hardware.

Next: Offline document search with RAG

Build prices come from our build datasheets: GPUs and devices are sourced retail or sold prices; other parts are sample prices until sourced, and the flag on each total says which. Model links go to our datasheets, where each parameter count and licence was checked against the Hugging Face API.