Fastest rig for Qwen2.5-Coder-32B (local)
Qwen2.5-Coder-32B is the strongest fully-open dense coding model that fits a single prosumer card at 4-bit. Here's the fastest decode tok/s submitted across every GPU and Apple Silicon tier.
Among measured rigs, the RTX 5090 is fastest for Qwen2.5-Coder-32B at 71.9 tok/s (llama.cpp, r_nkbs6d3-d21), about twice the 34.5 tok/s measured on a 96GB M3 Ultra (MLX 4-bit, r_721b4bls_oq). An RTX 3090 run of the Ollama qwen2.5-coder Q4_K_M tag measured 69.2 tok/s (r_6ahy-dq0f_0), but that run reports a 14.8B model, not the 32B, so it isn't ranked here.
Fastest submitted run for Qwen2.5-Coder-32B: RTX 5090 (32GB) at 71.91tok/s.
Qwen2.5-Coder-32B at Q4_K_M needs ~20 GB of weights plus KV cache, which lands it on a single 24-32 GB GPU (RTX 5090 / 4090 / 3090), a two-card split, or any Apple Silicon Max or Ultra with unified memory. Decode speed is set by memory bandwidth once the model fits, so the ranking below is the honest picture: every rig we have a real submitted run for, fastest first, each row linking the signed run. If your config isn't here yet, run the suite and submit — the row appears next refresh.
Submitted benchmarks
| Hardware | decode tok/s | Workload | Run |
|---|---|---|---|
| RTX 5090 (32GB) | 71.91tok/s | chat-short | r_nkbs6d3-d21 |
| M3 Ultra (60-core GPU) | 34.48tok/s | chat-short | r_721b4bls_oq |
See also: All hardware · All models · Methodology