Skip to main contentThe Local Bench

gpt-oss-120b

OpenAI · Apache-2.0 · Datasheet TLB-M-0010 rev. 2026.10

Evidence varies by field · estimates and placeholders labelled

  • Data checked: 2026-10-12. Review interval shown below. Prices need review after 30 days; specifications and benchmark collections after 90 days. Dates describe the recorded check, not a live retailer quote.

    • Verified Checked against the primary source (for example the model’s config.json on Hugging Face).
    • Sourced Taken from a cited source; not yet independently checked.
    • Estimate Calculated with the published formulas; not a measurement.
    • Sample Placeholder data for layout; not a real figure.
    • Not yet measured No public, citable score found for this benchmark yet. We never fill the gap with a guess.

    Verified metadata is not a measured performance result. Only cited runs matching model, device, quant and recorded context can replace speed estimates. Estimates assume one stream and F16 KV cache, with a ±25% uncertainty band. A published run applies only to its recorded runtime and conditions; missing settings are unknown.

    • gpt-oss-120b · dgx-spark-128g
      llama.cpp (CUDA) · MXFP4 · tg32 @ depth 0
      Recorded 2026-10-12 · historical source record · repeat count unknownSourced
      llama-bench -fa 1 -p 2048 -n 32 -ub 2048, ggml-org GGUF, commit 3795cc1e8 (2026-02-05), run by the llama.cpp maintainers on a DGX Spark. Run source
    • gpt-oss-120b · dgx-spark-128g
      llama.cpp (CUDA) · MXFP4 · tg32 @ depth 4096
      Recorded 2026-10-12 · historical source record · repeat count unknownSourced
      llama-bench -fa 1 -p 2048 -n 32 -ub 2048, ggml-org GGUF, commit 3795cc1e8 (2026-02-05), run by the llama.cpp maintainers on a DGX Spark. Run source
    • gpt-oss-120b · dgx-spark-128g
      llama.cpp (CUDA) · MXFP4 · tg32 @ depth 8192
      Recorded 2026-10-12 · historical source record · repeat count unknownSourced
      llama-bench -fa 1 -p 2048 -n 32 -ub 2048, ggml-org GGUF, commit 3795cc1e8 (2026-02-05), run by the llama.cpp maintainers on a DGX Spark. Run source
    • gpt-oss-120b · dgx-spark-128g
      llama.cpp (CUDA) · MXFP4 · tg32 @ depth 16384
      Recorded 2026-10-12 · historical source record · repeat count unknownSourced
      llama-bench -fa 1 -p 2048 -n 32 -ub 2048, ggml-org GGUF, commit 3795cc1e8 (2026-02-05), run by the llama.cpp maintainers on a DGX Spark. Run source
    • gpt-oss-120b · dgx-spark-128g
      llama.cpp (CUDA) · MXFP4 · tg32 @ depth 32768
      Recorded 2026-10-12 · historical source record · repeat count unknownSourced
      llama-bench -fa 1 -p 2048 -n 32 -ub 2048, ggml-org GGUF, commit 3795cc1e8 (2026-02-05), run by the llama.cpp maintainers on a DGX Spark. Run source
    • gpt-oss-120b · dgx-spark-128g
      llama.cpp (CUDA) · MXFP4 · tg32 @ 8k depth
      Recorded 2026-10-12 · historical source record · repeat count unknownSourced
      llama-bench -fa 1 -p 2048 -n 32 -ub 2048, ggml-org GGUF, commit 3795cc1e8 (2026-02-05), run by the llama.cpp maintainers on a DGX Spark. Run source

    Evidence labels · Freshness policy · Calculation methodology

116.8B
parameters, 5.1B active (MoE)
128K
token context window
64.3GB
memory at MXFP4, 8k context
17.3tok/s
decode on a used RTX 3090Estimate
—
KL-divergence at MXFP4 vs BF16Sample

At a glance

The model that sells 128 GB unified-memory boxes: 63 GB of MXFP4 weights, but only 5.1B parameters read per token.

At MXFP4 and 8k context it needs about 64.3 GB: 63.4 GB of weights, 0.3 GB of KV cache and 0.6 GB of runtime overhead. Only 5.1B of its 116.8B parameters are read per token, so it decodes far faster than its size suggests (see memory bandwidth). 18 of its 36 layers use a 128-token sliding window, which keeps the cache small at long context.

Estimated setup

Quant
MXFP4
Smallest standard card size at 8k
96 GB
Max context, 24 GB card
does not fit
Runtimes
llama.cpp, Ollama, LM Studio, MLX
Check it on your hardware

1 Estimates: decode ≈ η × bandwidth ÷ (active weights + KV cache), η 0.60 discrete / 0.45 unified / 0.50 CPU (concept brief §7.3). Not measurements.

2 Fit rules from the concept brief §7.3, including the 0.75 unified-memory factor used by canirun.ai.

Flags: Verified = checked against config.json / HF API; Sourced = from a cited source; Estimate = formula; Sample = placeholder. Markdown · JSON · Report a problem