# GeForce RTX 4090 24 GB

> Fastest 24 GB card; the extra compute shows up in prefill more than decode.

Canonical: https://thelocalbench.com/hardware/rtx-4090-24g · Data as of 2026-10-12 · The Local Bench (thelocalbench.com), data as of 2026-10-12, CC BY 4.0

Flags: [verified] checked against primary source · [sourced] cited · [estimate] formula · [sample] placeholder

## Specifications

| Field | Value | Flag | Source / note |
|---|---|---|---|
| memory_gb | 24 | sourced | https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/ |
| bandwidth_gbs | 1008 | sourced | https://en.wikipedia.org/wiki/GeForce_RTX_40_series |
| fp16_tflops | 165 | sample | Sample FP16 dense compute; pending vendor-sheet citation. |
| load_watts | 500 | sample | Sample wall power under load. |
| price_usd | 4395 | sourced | https://www.newegg.com/p/pl?d=rtx+4090+24gb+graphics+card&N=4131 |

## Models that run (default quant, 8k)

| Model | Quant | Memory GB | Verdict | Decode tok/s |
|---|---|---|---|---|
| [Llama 3.1 8B Instruct](/models/llama-3.1-8b.md) | Q4_K_M | 6.6 | fit | 100.9 [estimate] |
| [Qwen3 8B](/models/qwen3-8b.md) | Q4_K_M | 6.8 | fit | 97.0 [estimate] |
| [Qwen3 14B](/models/qwen3-14b.md) | Q4_K_M | 10.9 | fit | 58.5 [estimate] |
| [gpt-oss-20b](/models/gpt-oss-20b.md) | MXFP4 | 12.9 | fit | 264.2 [estimate] |
| [Mistral Small 3.2 24B](/models/mistral-small-3.2-24b.md) | Q4_K_M | 16.2 | fit | 38.6 [estimate] |
| [Gemma 3 27B IT](/models/gemma-3-27b.md) | Q4_K_M | 18.2 | fit | 34.3 [estimate] |
| [Qwen3 30B-A3B](/models/qwen3-30b-a3b.md) | Q4_K_M | 19.9 | fit | 215.1 [estimate] |
| [Qwen3 32B](/models/qwen3-32b.md) | Q4_K_M | 22.5 | tight | 27.6 [estimate] |
| [Llama 3.3 70B Instruct](/models/llama-3.3-70b.md) | Q4_K_M | 45.8 | offload | 1.4 [estimate] |
| [gpt-oss-120b](/models/gpt-oss-120b.md) | MXFP4 | 64.3 | offload | 17.3 [estimate] |