GeForce RTX 4090 24 GB
Data checked: 2026-10-12. Review interval shown below. Prices need review after 30 days; specifications and benchmark collections after 90 days. Dates describe the recorded check, not a live retailer quote.
- Verified Checked against the primary source (for example the model’s config.json on Hugging Face).
- Sourced Taken from a cited source; not yet independently checked.
- Estimate Calculated with the published formulas; not a measurement.
- Sample Placeholder data for layout; not a real figure.
- Not yet measured No public, citable score found for this benchmark yet. We never fill the gap with a guess.
Verified metadata is not a measured performance result. Only cited runs matching model, device, quant and recorded context can replace speed estimates. Estimates assume one stream and F16 KV cache, with a ±25% uncertainty band. A published run applies only to its recorded runtime and conditions; missing settings are unknown.
- llama-2-7b · rtx-4090-24g
llama.cpp (CUDA) · Q4_0 · tg128
Recorded 2026-10-12 · historical source record · repeat count unknownSourced
Llama 2 7B Q4_0, llama-bench tg128, full GPU offload, flash attention off, single GPU. Community scoreboard entry; commit and submitter in the source table. Run source
Evidence labels · Freshness policy · Calculation methodology
Fastest 24 GB card; the extra compute shows up in prefill more than decode.
Decode-speed ceiling for a model is 1,008 GB/s ÷ GB read per token. For a 16.6 GB file that is 61 tok/s; real runtimes reach roughly 60% of it. Why bandwidth is the limit.
Specifications
Memory | 24 GBSourcedVendor spec page, checked 2026-10-12. Source |
|---|---|
Bandwidth | 1,008 GB/sSourcedNVIDIA's GeForce spec pages list bus width but not bandwidth; figure from the cited Wikipedia spec table (bus width cross-checked on nvidia.com, 2026-10-12). Source |
FP16 dense compute | 165 TFLOPSSampleSample FP16 dense compute; pending vendor-sheet citation. |
Host RAM (offload) | 64 GBSampleAssumed host RAM for offload. |
Vendor TDP / board power | 450 WSourcedTotal Graphics Power, nvidia.com. Source |
Load power at wall | 500 WSampleSample wall power under load. |
Price, USD | $4,395SourcedLowest new in-stock RTX 4090 listing on Newegg US (incl. marketplace sellers; card discontinued, scarce). Checked 2026-10-12. Source |
Price, CAD | CA$5,600SourcedLowest new in-stock listing on Newegg Canada (MSI Gaming Trio, ships from Canada). Checked 2026-10-12. Source |
Models that run
Default quant, single stream, 8k context. Fastest first; estimates are ±25% until measured.
Model | Quant | Memory | Verdict | Decode, tok/s | Max context |
|---|---|---|---|---|---|
| gpt-oss-20b | MXFP4 | 12.9 GB | Fits | 264.2Estimate | 128k |
| Qwen3 30B-A3B | Q4_K_M | 19.9 GB | Fits | 215.1Estimate | 32k |
| Llama 3.1 8B Instruct | Q4_K_M | 6.6 GB | Fits | 100.9Estimate | 64k |
| Qwen3 8B | Q4_K_M | 6.8 GB | Fits | 97.0Estimate | 32k |
| Qwen3 14B | Q4_K_M | 10.9 GB | Fits | 58.5Estimate | 32k |
| Mistral Small 3.2 24B | Q4_K_M | 16.2 GB | Fits | 38.6Estimate | 32k |
| Gemma 3 27B IT | Q4_K_M | 18.2 GB | Fits | 34.3Estimate | 64k |
| Qwen3 32B | Q4_K_M | 22.5 GB | Tight | 27.6Estimate | 8k |
| gpt-oss-120b | MXFP4 | 64.3 GB | Needs offload | 17.3Estimate | — |
| Llama 3.3 70B Instruct | Q4_K_M | 45.8 GB | Needs offload | 1.4Estimate | — |
Published reference runs
Third-party llama.cpp measurements on this device, with conditions and source. Llama 2 7B Q4_0 runs are the community calibration test; they are not catalogue models.
Model | Quant | Date | Runtime · test | Decode, tok/s | Conditions |
|---|---|---|---|---|---|
| llama-2-7b | Q4_0 | 2026-10-12 | llama.cpp (CUDA) · tg128 | 186.2Sourced | Llama 2 7B Q4_0, llama-bench tg128, full GPU offload, flash attention off, single GPU. Community scoreboard entry; commit and submitter in the source table. Source |