# Mistral Small 3.2 24B datasheet

> A dense 24B with vision input and a 128K window. Q4_K_M is 14.3 GB, a comfortable fit on a 24 GB card at long context.

Canonical: https://thelocalbench.com/models/mistral-small-3.2-24b · Data as of 2026-10-12 · The Local Bench (thelocalbench.com), data as of 2026-10-12, CC BY 4.0

Flags: [verified] checked against primary source · [sourced] cited · [estimate] formula · [sample] placeholder

## Specifications

| Field | Value | Flag | Source |
|---|---|---|---|
| params_total | 24.01 B | verified | https://huggingface.co/api/models/mistralai/Mistral-Small-3.2-24B-Instruct-2506 |
| params_active | 24.01 B | verified | https://huggingface.co/api/models/mistralai/Mistral-Small-3.2-24B-Instruct-2506 |
| layers | 40 | verified | https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/blob/main/config.json |
| kv_heads | 8 | verified | https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/blob/main/config.json |
| q_heads | 32 | verified | https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/blob/main/config.json |
| head_dim | 128 | verified | https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/blob/main/config.json |
| hidden_size | 5120 | verified | https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/blob/main/config.json |
| context | 131072 tokens | verified | https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/blob/main/config.json |
| license | Apache-2.0 | verified | https://www.apache.org/licenses/LICENSE-2.0 |

## Memory and quants (total = weights + F16 KV + overhead)

| Quant | bpw | File GB | At 8k | At 32k | At 128k | KLD | 24 GB card, 8k |
|---|---|---|---|---|---|---|---|
| Q3_K_M | 3.91 | 11.47 [verified] | 13.4 | 17.6 | 34.4 | 0.071 [sample] | fit |
| Q4_K_M | 4.85 | 14.33 [verified] | 16.2 | 20.4 | 37.3 | 0.017 [sample] | fit |
| Q5_K_M | 5.69 | 16.76 [verified] | 18.7 | 22.9 | 39.7 | 0.008 [sample] | fit |
| Q6_K | 6.56 | 19.35 [verified] | 21.3 | 25.5 | 42.3 | 0.004 [sample] | tight |
| Q8_0 | 8.5 | 25.05 [verified] | 27.0 | 31.2 | 48.0 | 0.001 [sample] | offload |
| F16 | 16 | 47.15 [verified] | 49.1 | 53.3 | 70.1 | — | offload |

## Speed (default quant, 8k context, single stream)

| Device | Verdict | Decode tok/s | Flag |
|---|---|---|---|
| [GeForce RTX 5060 Ti 16 GB](/hardware/rtx-5060-ti-16g.md) | offload | 12.7 | estimate |
| [GeForce RTX 4060 Ti 16 GB](/hardware/rtx-4060-ti-16g.md) | offload | 9.1 | estimate |
| [Radeon RX 7900 XTX 24 GB](/hardware/rx-7900-xtx-24g.md) | fit | 36.8 | estimate |
| [GeForce RTX 3090 24 GB (used)](/hardware/rtx-3090-24g-used.md) | fit | 35.8 | estimate |
| [CPU only · Ryzen 9 9950X + 128 GB DDR5](/hardware/cpu-ryzen-9-9950x-128g.md) | fit | 2.9 | estimate |
| [2× RTX 3090 (used) 48 GB](/hardware/2x-rtx-3090-48g-used.md) | fit | 35.8 | estimate |
| [Mac Studio M5 Max (base)](/hardware/mac-studio-m5-max.md) | fit | 13.2 | estimate |
| [Ryzen AI Max+ 395 mini PC 128 GB](/hardware/ryzen-ai-max-395-128g.md) | fit | 7.4 | estimate |
| [GeForce RTX 4090 24 GB](/hardware/rtx-4090-24g.md) | fit | 38.6 | estimate |
| [NVIDIA DGX Spark FE 128 GB](/hardware/dgx-spark-128g.md) | fit | 7.8 | estimate |
| [MacBook Pro M4 Max 40c · 128 GB](/hardware/macbook-pro-m4-max-128g.md) | fit | 15.7 | estimate |
| [Mac Studio M5 Ultra (base)](/hardware/mac-studio-m5-ultra.md) | fit | 34.5 | estimate |
| [GeForce RTX 5090 32 GB](/hardware/rtx-5090-32g.md) | fit | 68.6 | estimate |
| [Mac Studio M5 Ultra · 512 GB](/hardware/mac-studio-m5-ultra-512g.md) | fit | 34.5 | estimate |

## Quality

LMArena text score: 1338.1 [sourced] (LMArena text, overall, entry 'mistral-small-2506': 1338.1 (95% CI 1332.9–1343.3, 17,466 votes), leaderboard published 2026-10-08. CC-BY-4.0.)

JSON: https://thelocalbench.com/api/v1/models/mistral-small-3.2-24b.json · Configurator: https://thelocalbench.com/configurator?model=mistral-small-3.2-24b