Skip to main contentThe Local Bench

Use case 9 of 13

Answering the phone for a small business

A voice agent that answers, takes messages, gives opening hours and books appointments, running on hardware in the back office.

A blue telephone handset with coiled cord beside a black mini PC and a business card
Image: generated with Grok Imagine

The real example

Pipecat voice agents

Pipecat is an open-source (BSD-2-Clause) Python framework, maintained by Daily and the community, for real-time voice and multimodal conversational agents; its README lists customer intake and support bots among its uses. It chains speech recognition, a language model and text-to-speech with turn-taking. Parakeet TDT 0.6B v2, a strong local speech recogniser, has a real-time factor above 6,000 on the Open ASR Leaderboard, so transcription is not the bottleneck. For calendar and CRM actions, BFCL V4 is a useful guide to which small models call tools reliably.

Sources: Pipecat on GitHub · Open ASR Leaderboard · Berkeley Function-Calling Leaderboard

Why run it locally

Callers share names, numbers and reasons for calling. Local models mean no per-minute AI fees, which matters on long, quiet hours.

Recommended stack

Three tiers built from our builds. Weights-in-memory is our arithmetic (parameters × bits), checked against each device's usable memory; it excludes KV cache and runtime overhead. Runtime compatibility is not validated.

Mid

Used-3090 24 GB

$1,980Sample

24 GB on GeForce RTX 3090 24 GB (used)

Known weights ≈ 4.9 GB; some model sizes are unknown. Memory fit cannot be determined.Estimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

A tool-calling 8B for bookings. xLAM-2 is non-commercial, so check the licence; Qwen3 8B is an Apache-2.0 alternative.

Runtime: Pipecat + vLLM

Pro

Dual-GPU 48 GB

$4,044Sample

48 GB on 2× RTX 3090 (used) 48 GB

Known weights ≈ 24.9 GB; some model sizes are unknown. Memory fit cannot be determined.Estimate

Excludes KV cache and runtime overhead. Runtime compatibility is not validated.

Several concurrent calls, a larger model for open questions.

Runtime: Pipecat + vLLM

Setup outline

  1. Pick a telephony path. Pipecat connects to hosted SIP and phone services; a fully local PBX is possible but more work.
  2. Run speech-to-text, the LLM and TTS as local services, and wire them into a Pipecat pipeline.
  3. Give the agent a short script, opening hours and a single booking tool with strict input checks.
  4. Record consent prompts if your jurisdiction requires them, and always offer a human fallback.

Limits

  • The phone line itself usually goes through a carrier, so calls are only as private as that leg.
  • Latency matters more than intelligence. Measure time-to-first-audio on your hardware.
  • Disclose that callers are talking to an automated system where the law requires it.

Next: Teaching a robot arm at home

Build prices come from our build datasheets: GPUs and devices are sourced retail or sold prices; other parts are sample prices until sourced, and the flag on each total says which. Model links go to our datasheets, where each parameter count and licence was checked against the Hugging Face API.