gpt-oss-20b-MXFP4-Q4
18 workload results across 5 hardware configurations.
Fastest local config
324.2 decode tok/s
on RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB via llama.cpp (Q4_K_M). see full run
For a standard Q4_K_M quantization using llama.cpp, here are measured results for short-chat generation.
On llama.cpp Q4_K_M, gpt-oss-20b is fastest on the RTX 5090 at 324.2 tok/s, 34% faster than the RTX 4090 (242.6 tok/s); slowest: RTX 3090 at 207.2 tok/s.
| Card | Decode tok/s | TTFT (166-token prompt) | Run |
|---|---|---|---|
| RTX 5090 | 324.2 (single run) | 283 ms | r_dwp5fgdapmw |
| RTX 4090 | 242.6 (single run) | 280 ms | r_t-a9v_ipdit |
| RTX 3090 | 207.2 (single run) | 252 ms | r_nakw1nq6clw |
gpt-oss-20b, Q4_K_M, llama.cpp, chat-short, batch 1. Only cards with their standard memory are listed; CPU-only hosts are excluded. Runs use different llama.cpp versions (RTX 5090: 0.7.0-dev (build 1, commit 1268eda); RTX 4090: 0.6.0-dev (build 1, commit df18bee); RTX 3090: 0.7.0-dev (build 1, commit ea0c4c2)), so treat this as measured reference points, not a controlled ranking.
Local runs (18 workload results)
Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.
RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 324.2tok/s | 586.8tok/s | 283ms | r_dwp5fgdapmw |
| chat-long | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 311.0tok/s | 12,638.4tok/s | 252ms | r_dwp5fgdapmw |
| concurrent-decode | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 322.5tok/s | no data | no data | r_dwp5fgdapmw |
| agent-trace | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 319.6tok/s | 23,887.0tok/s | 89.2ms | r_dwp5fgdapmw |
RTX 3090 (24GB) + AMD EPYC 7282 16-Core Processor (16c) + 126GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.7.0-dev (build 1, commit ea0c4c2) | Q4_K_M | 207.2tok/s | 658.3tok/s | 252ms | r_nakw1nq6clw |
| chat-long | llama.cpp@0.7.0-dev (build 1, commit ea0c4c2) | Q4_K_M | 199.8tok/s | 4,606.3tok/s | 692ms | r_nakw1nq6clw |
| concurrent-decode | llama.cpp@0.7.0-dev (build 1, commit ea0c4c2) | Q4_K_M | 205.1tok/s | no data | no data | r_nakw1nq6clw |
| agent-trace | llama.cpp@0.7.0-dev (build 1, commit ea0c4c2) | Q4_K_M | 202.2tok/s | 9,892.7tok/s | 224ms | r_nakw1nq6clw |
RTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 242.6tok/s | 593.8tok/s | 280ms | r_t-a9v_ipdit |
| chat-long | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 231.2tok/s | 7,864.1tok/s | 405ms | r_t-a9v_ipdit |
| concurrent-decode | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 239.6tok/s | no data | no data | r_t-a9v_ipdit |
| agent-trace | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 236.1tok/s | 13,439.2tok/s | 150ms | r_t-a9v_ipdit |
RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp | - | 318.4tok/s | no data | 173ms | r_b9ul-vxh9sc |
| chat-long | llama.cpp | - | 301.6tok/s | no data | 239ms | r_b9ul-vxh9sc |
| concurrent-decode | llama.cpp | - | 302.8tok/s | no data | no data | r_b9ul-vxh9sc |
| agent-trace | llama.cpp | - | 304.5tok/s | 26,907.5tok/s | 77.7ms | r_b9ul-vxh9sc |
| chat-short | llama.cpp | - | 69.42tok/s | no data | 333ms | r_r9h57uts9lr |
M3 Ultra (60-core GPU) + 96GB unified
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | mlx@0.31.3 | - | 152.7tok/s | 239.9tok/s | 692ms | r_3ijun8ltjnb |
Community folklore
23 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.
- communityconfidence 60%
221.0tok/s — GPT-OSS-20B on RTX 5090 via lm-studio
our signed data: RTX 5090 · GPT-OSS-20B
“GPT-OSS-20B on RTX 5090 – 221 tok/s in LM Studio (default settings + FlashAttention) Just tested **GPT-OSS-20B** locally using **LM Studio v0.3.21-b4** on my machine with an”
- communityconfidence 60%
72.00tok/s — GPT OSS 20B on M4 Pro via mlx
our signed data: M4 Pro · GPT OSS 20B
“edge frozen as of 2024-06, handled rescaling and converting a recipe into metric that Qwen 3 30B 3AB 2507 completely hosed, churns at about 72 tok/s on a MacBook M4 Pro Max 4 bit MLX. And it ALMOST got a side scrolling shooter working, whereas the 120B didn't, and Qwen 3 didn't …”
- communityconfidence 60%
103.0tok/s — gpt-oss-20b on M4 Max via mlx
our signed data: M4 Max · gpt-oss-20b
“/gpt-oss-20b-GGUF]() (thanks [unsloth]() 🙌) The qualities are roughly the same for our use case, but performance: On an M4 Max, MLX hit \~103 tok/s, about 25% faster than GGUF. # Quick tip You can paste any Hugging Face repo name into the CLI and pull it directly: nexa …”
- communityconfidence 60%
180.0tok/s — gpt oss 20b on Radeon RX 9070 XT via llama.cpp
our signed data: Radeon RX 9070 XT · gpt oss 20b
“hanics) gpt OSs 20v won by a metric mile. It did run faster than gpt OSs though on llama.cpp (9070xt) with gpt OSs at 140 tok/s and lfm at 180 tok/s”
- communityconfidence 60%
140.0tok/s — gpt oss 20b on Radeon RX 9070 XT via llama.cpp
our signed data: Radeon RX 9070 XT · gpt oss 20b
“cs and structural mechanics) gpt OSs 20v won by a metric mile. It did run faster than gpt OSs though on llama.cpp (9070xt) with gpt OSs at 140 tok/s and lfm at 180 tok/s”
- communityconfidence 60%
120.0tok/s — gpt-oss-20b on RTX 3090 via llama.cpp
our signed data: RTX 3090 · gpt-oss-20b
“i haven't run 120b, but ran gpt-oss-20b q4/q8 on a 3090 nd saw ~70–120 tokens/sec depending on threads, context length and quant. 395 will fit bigger layouts but limited memory bandwidth often cuts sustained tps, so repro”
- communityconfidence 60%
400.0tok/s — gpt-oss-20b on RTX 5090 via vllm
our signed data: RTX 5090 · gpt-oss-20b
“UDA\_ARCH=120a / Tried VLLM also * Model: gpt-oss-20b MXFP4 (GGUF) / Safetensors **The Problem:** * First 1-2 inferences work fine (\~300-400 tok/s) * On the 3rd or 4th inference, GPU hangs with "CUDA error: illegal memory access" * GPU enters error state, fans spin to 100% * n…”
- communityconfidence 55%
30.00tok/s — gpt-oss-20b via vllm BF16
our signed data: gpt-oss-20b
“M compatible, we rewrote Transformers inference for 3× faster speeds (\~21 tok/s). For BF16, Unsloth also delivers the fastest inference (\~30 tok/s), especially relative to VRAM use vs. any other implementation. 2. We made a free & completely new custom notebook showing how RL c…”
- communityconfidence 55%
21.00tok/s — gpt-oss-20b via vllm BF16
our signed data: gpt-oss-20b
“1. Inference is crucial in RL training. Since gpt-oss RL isn’t vLLM compatible, we rewrote Transformers inference for 3× faster speeds (\~21 tok/s). For BF16, Unsloth also delivers the fastest inference (\~30 tok/s), especially relative to VRAM use vs. any other implementation. 2…”
- communityconfidence 55%
30.00tok/s — gpt-oss-20b on Ryzen AI MAX+ int8
our signed data: Ryzen AI MAX+ · gpt-oss-20b
“d.com/en/blogs/2025/how-to-run-openai-gpt-oss-20b-120b-models-on-amd-ryzen-ai-radeon.html) and and there's this from a month a go claiming "30 tokens per second" (not clear if 120b or 20b). I can't tell if the flops are int8 flops of bf16 or fp16 on the 395. In theory if we assum…”
gpt-oss-20b-MXFP4-Q4 on hardware
Request a gpt-oss-20b-MXFP4-Q4 run
No measured run yet for these pairs. Vote for the one you want; the most-requested runs go first. See all requests.
We never sell or share your email. Privacy