RTX 5090 LLM benchmark
On the RTX 5090, measured single-user chat decode speeds are: ~7–9B models at 229.4 tok/s (llama.cpp; Measured, middle of 10 results, source run), ~12–14B models at 137.3 tok/s (Q4_0, llama.cpp; Measured, middle of 10 results, source run) and ~22–32B models at 69.6 tok/s (llama.cpp; Measured, middle of 121 results, source run). Short-chat workload, local runtimes, dense models only; quantization, backend and settings vary between submissions, so check each row in the tables below.
Specs vendor specs, not measured
- VRAM
- 32 GB
- Memory bandwidth
- 1792 GB/s
- Launch date
- 2025-01-30
Sources: nvidia.com, nvidianews.nvidia.com, nvidia.com. Vendor-published figures, accessed 2026-10-14.
Fastest single result in these submissions (any model size)
341.9 decode tok/s
stable-code-instruct-3b via llama.cpp. Workload: chat-short. Reported hardware: RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB. see full run
Original measurements · 5 September 2026
Gemma 4 12B vs Qwen3.8-27B on RTX 5090
Compare response speed, first-token delay and GPU allocation on the same rig. Both tested model files fit entirely on this 32 GB GPU with a 16,384-token context configured. The prompts below are shorter than that configured limit.
Read the comparison, exact tested files and downloadable results →
Gemma 4 12B IT QAT
Q4_0 · 6.98 GB model file
7.75 GiB runtime GPU allocation
Model + context + compute buffers; all 49 layers on GPU.
Settings and three source runs →Qwen3.8-27B
Q4_K_M · 18.97 GB model file
18.62 GiB runtime GPU allocation
Model + context + compute buffers; all 65 layers on GPU.
Settings and three source runs →| Model / workload | Decode tok/s ↑ | First token ms ↓ | Actual input tokens | Actual output tokens |
|---|---|---|---|---|
| Gemma 4 12B IT QATQ4_0 · chat-short | 139.5(137.3–141.1) | 59.7(52.2–61.8) | 121 | 256 |
| Qwen3.8-27BQ4_K_M · chat-short | 65.7(65.4–65.7) | 117.5(117.1–131.7) | 114 | 256 |
| Gemma 4 12B IT QATQ4_0 · chat-long | 135.6(135.6–137.1) | 619.3(577.2–622.9) | 3,188 | 620 |
| Qwen3.8-27BQ4_K_M · chat-long | 65.1(65.1–65.1) | 909.4(908.0–910.5) | 3,184 | 874 |
Same llama.cpp CUDA runtime, suite-v1 text prompts, one request at a time, warm models, thinking and prompt caching off. Output caps: 256 tokens for chat-short and 1,024 for chat-long. Tokenizers and actual output lengths differ.
Use these results to shortlist models for your latency and memory needs, then test answer quality on your own tasks. Coding accuracy, reasoning quality, vision and multi-user capacity were not measured. Runtime allocation is not peak whole-device memory; other idle services remained GPU-resident.
RTX 5090 model studies
Repeated tests with pinned model files, runtime settings, observed ranges and source runs. Choose a study for configuration details beyond the leaderboard's single result.
Qwen2.5-Coder-14B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 137.3tok/s | 2,016.9tok/s | 65.0ms | r_xnr_k4emoyr |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 128.9tok/s | 5,987.2tok/s | 529ms | r_xnr_k4emoyr |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 136.7tok/s | no data | no data | r_xnr_k4emoyr |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 128.2tok/s | 18,881.9tok/s | 113ms | r_xnr_k4emoyr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 140.0tok/s | no data | 42.1ms | r_opuj21f13-_ |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 123.0tok/s | no data | 457ms | r_opuj21f13-_ |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 133.8tok/s | no data | no data | r_opuj21f13-_ |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 130.5tok/s | 21,985.9tok/s | 104ms | r_opuj21f13-_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 136.0tok/s | no data | 51.3ms | r_p_f63tcgans |
Qwen3-Coder-30B-A3B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 304.9tok/s | 514.5tok/s | 214ms | r_vsype4pm1ko |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 266.2tok/s | 8,353.2tok/s | 377ms | r_vsype4pm1ko |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 296.7tok/s | no data | no data | r_vsype4pm1ko |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 278.8tok/s | 23,318.7tok/s | 96.6ms | r_vsype4pm1ko |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.41tok/s | no data | 334ms | r_pm_a1uf2ufc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.55tok/s | no data | 1,204ms | r_pm_a1uf2ufc |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.04tok/s | no data | no data | r_pm_a1uf2ufc |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.09tok/s | 4,816.7tok/s | 451ms | r_pm_a1uf2ufc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 259.9tok/s | no data | 218ms | r_c7qyvvmmsv1 |
gpt-oss-20b
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 324.2tok/s | 586.8tok/s | 283ms | r_dwp5fgdapmw |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 311.0tok/s | 12,638.4tok/s | 252ms | r_dwp5fgdapmw |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 322.5tok/s | no data | no data | r_dwp5fgdapmw |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 319.6tok/s | 23,887.0tok/s | 89.2ms | r_dwp5fgdapmw |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 318.4tok/s | no data | 173ms | r_b9ul-vxh9sc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 301.6tok/s | no data | 239ms | r_b9ul-vxh9sc |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 302.8tok/s | no data | no data | r_b9ul-vxh9sc |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 304.5tok/s | 26,907.5tok/s | 77.7ms | r_b9ul-vxh9sc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.42tok/s | no data | 333ms | r_r9h57uts9lr |
Qwen3.6-27B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 75.09tok/s | 302.2tok/s | 371ms | r_pob-co51bd- |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 74.55tok/s | 2,644.8tok/s | 1,203ms | r_pob-co51bd- |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 75.13tok/s | no data | no data | r_pob-co51bd- |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 74.13tok/s | 4,464.5tok/s | 485ms | r_pob-co51bd- |
Qwen3-32B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 69.83tok/s | 1,047.0tok/s | 105ms | r_vhbqyt1f_xr |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 64.13tok/s | 3,139.6tok/s | 1,002ms | r_vhbqyt1f_xr |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 68.52tok/s | no data | no data | r_vhbqyt1f_xr |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 66.46tok/s | 8,161.5tok/s | 253ms | r_vhbqyt1f_xr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.51tok/s | no data | 158ms | r_-txe_hiq44n |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 62.30tok/s | no data | 1,067ms | r_-txe_hiq44n |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.64tok/s | no data | no data | r_-txe_hiq44n |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 64.34tok/s | 8,394.1tok/s | 259ms | r_-txe_hiq44n |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.45tok/s | no data | 327ms | r_phvxm9dcak0 |
Qwen3.8-27B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 75.19tok/s | 563.4tok/s | 273ms | r_qfrb2zmyk0k |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 74.57tok/s | 2,499.5tok/s | 1,290ms | r_qfrb2zmyk0k |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 75.40tok/s | no data | no data | r_qfrb2zmyk0k |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 73.52tok/s | 5,065.3tok/s | 436ms | r_qfrb2zmyk0k |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.66tok/s | 969.8tok/s | 118ms | r_dz5a3lzxhrj |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.10tok/s | 3,496.8tok/s | 911ms | r_dz5a3lzxhrj |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.73tok/s | 973.6tok/s | 117ms | r_c3ps9wjygi9 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.09tok/s | 3,506.5tok/s | 908ms | r_c3ps9wjygi9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.43tok/s | 865.3tok/s | 132ms | r_7mod38qsldj |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.10tok/s | 3,501.1tok/s | 909ms | r_7mod38qsldj |
Nemotron-3.5-Bolt-74B-A9B-24GB-ctx32k
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 157.3tok/s | 355.4tok/s | 338ms | r_ymx58emedxd |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 155.7tok/s | 2,605.8tok/s | 1,247ms | r_ymx58emedxd |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 157.7tok/s | no data | no data | r_ymx58emedxd |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 154.9tok/s | 5,095.7tok/s | 431ms | r_ymx58emedxd |
Nemotron-3.5-Bolt-74B-A9B-24GB
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 154.7tok/s | 353.4tok/s | 340ms | r_z-l1yelv_67 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 154.2tok/s | 2,623.3tok/s | 1,239ms | r_z-l1yelv_67 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 156.0tok/s | no data | no data | r_z-l1yelv_67 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 149.3tok/s | 5,097.2tok/s | 418ms | r_z-l1yelv_67 |
LFM2.5-24B-A2B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 308.5tok/s | 553.1tok/s | 206ms | r_c8k6fleym09 |
| chat-long | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 300.4tok/s | 12,418.3tok/s | 264ms | r_c8k6fleym09 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 321.3tok/s | no data | no data | r_c8k6fleym09 |
| agent-trace | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 313.2tok/s | 18,108.8tok/s | 117ms | r_c8k6fleym09 |
gemma-4-12b-it-qat
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| long-context-decay | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 142.6tok/s | 7,115.8tok/s | 3,355ms | r_-lh-vczf7ik |
| long-context-decay | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 136.3tok/s | 7,161.0tok/s | 3,334ms | r_1srqmpjnkgd |
| long-context-decay | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 134.2tok/s | 6,022.4tok/s | 3,964ms | r_8q-uf-rq-gp |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 141.1tok/s | 2,027.9tok/s | 59.7ms | r_v5arqmazf31 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 135.6tok/s | 5,147.7tok/s | 619ms | r_v5arqmazf31 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 139.5tok/s | 1,958.9tok/s | 61.8ms | r_c2podhe9qwc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 137.1tok/s | 5,118.0tok/s | 623ms | r_c2podhe9qwc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 137.3tok/s | 2,316.4tok/s | 52.2ms | r_8xmm65n1fhq |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 135.6tok/s | 5,523.3tok/s | 577ms | r_8xmm65n1fhq |
Coder-V2-Lite-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 293.1tok/s | no data | 297ms | r_bfpto9so2o1 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 181.2tok/s | no data | 361ms | r_bfpto9so2o1 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 268.4tok/s | no data | no data | r_bfpto9so2o1 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 204.5tok/s | 24,978.7tok/s | 80.7ms | r_bfpto9so2o1 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 309.5tok/s | no data | 268ms | r_0_gs1rgl2fl |
Yi-Coder-9B-Chat
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.79tok/s | no data | 317ms | r_zlh6az5q0o_ |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.60tok/s | no data | 1,195ms | r_zlh6az5q0o_ |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.99tok/s | no data | no data | r_zlh6az5q0o_ |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.76tok/s | 4,478.0tok/s | 397ms | r_zlh6az5q0o_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 199.3tok/s | no data | 36.2ms | r_u4iojm6-ekg |
stable-code-instruct-3b
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 341.9tok/s | no data | 175ms | r_q9f15lz6831 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 289.1tok/s | no data | 232ms | r_q9f15lz6831 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 356.1tok/s | no data | no data | r_q9f15lz6831 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 292.8tok/s | 44,387.5tok/s | 42.7ms | r_q9f15lz6831 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 331.0tok/s | no data | 36.7ms | r_8l57cim1i10 |
Qwen2.5-32B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.71tok/s | no data | 77.0ms | r_bjy5a5izxjc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 65.36tok/s | no data | 968ms | r_bjy5a5izxjc |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.60tok/s | no data | no data | r_bjy5a5izxjc |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.45tok/s | 10,407.9tok/s | 214ms | r_bjy5a5izxjc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.90tok/s | no data | 115ms | r_twfs86tf_xf |
Qwen2.5-14B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 130.3tok/s | no data | 48.6ms | r_xr4qdv1hgf2 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 122.7tok/s | no data | 489ms | r_xr4qdv1hgf2 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 131.4tok/s | no data | no data | r_xr4qdv1hgf2 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 122.8tok/s | 20,315.4tok/s | 111ms | r_xr4qdv1hgf2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 133.3tok/s | no data | 55.9ms | r_tj9bu7gvnvh |
Llama-3.1-8B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 229.4tok/s | no data | 35.6ms | r_qr4srge34da |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 211.4tok/s | no data | 266ms | r_qr4srge34da |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.0tok/s | no data | no data | r_qr4srge34da |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 210.3tok/s | 36,720.3tok/s | 60.8ms | r_qr4srge34da |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.2tok/s | no data | 33.8ms | r_kfrkg-vn376 |
Qwen2.5-7B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.7tok/s | no data | 37.0ms | r_1shiviswt3d |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.4tok/s | no data | 239ms | r_1shiviswt3d |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 240.7tok/s | no data | no data | r_1shiviswt3d |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.5tok/s | 38,007.7tok/s | 57.1ms | r_1shiviswt3d |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 246.7tok/s | no data | 33.9ms | r_3yn-4321hp- |
phi-4
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 139.1tok/s | no data | 51.1ms | r_k1u_k1j_1i2 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 133.7tok/s | no data | 433ms | r_k1u_k1j_1i2 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 140.2tok/s | no data | no data | r_k1u_k1j_1i2 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 124.8tok/s | 21,162.4tok/s | 102ms | r_k1u_k1j_1i2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 140.9tok/s | no data | 56.7ms | r_e-k4aea8ipr |
gemma-2-9b-it
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.45tok/s | no data | 325ms | r_1_xl4zb5-xj |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.42tok/s | no data | 1,044ms | r_1_xl4zb5-xj |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.57tok/s | no data | no data | r_1_xl4zb5-xj |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.46tok/s | 4,816.8tok/s | 420ms | r_1_xl4zb5-xj |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 152.7tok/s | no data | 58.9ms | r__b_bzmmab_8 |
Qwen2.5-Coder-32B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.41tok/s | no data | 384ms | r_v983y0y3r2u |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.15tok/s | no data | 1,251ms | r_v983y0y3r2u |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.96tok/s | no data | no data | r_v983y0y3r2u |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.56tok/s | 4,677.1tok/s | 414ms | r_v983y0y3r2u |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.91tok/s | no data | 71.0ms | r_nkbs6d3-d21 |
Qwen2.5-Coder-7B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 255.3tok/s | no data | 35.4ms | r_2b9o6y_49mi |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 244.0tok/s | no data | 228ms | r_2b9o6y_49mi |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 254.0tok/s | no data | no data | r_2b9o6y_49mi |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 229.9tok/s | 38,735.2tok/s | 57.4ms | r_2b9o6y_49mi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 244.9tok/s | no data | 36.1ms | r_mln72x5zbis |
Codestral-22B-v0.1
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 93.72tok/s | no data | 68.6ms | r_4q040m4scic |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 87.36tok/s | no data | 849ms | r_4q040m4scic |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 100.3tok/s | no data | no data | r_4q040m4scic |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 93.44tok/s | 10,351.0tok/s | 191ms | r_4q040m4scic |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.95tok/s | no data | 199ms | r_sqr8liqh4ii |
Qwen3.6-35B-A3B-Q4_K_M.gguf
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.3tok/s | no data | 263ms | r_dp6sr2_iwcf |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 224.0tok/s | no data | 167ms | r_a56-wxl21lk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.5tok/s | no data | 185ms | r_k070lz99uzi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 221.6tok/s | no data | 207ms | r_dqelw_a8f2c |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 221.8tok/s | no data | 204ms | r_c25tn-sl4bn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 208.6tok/s | no data | 220ms | r_d1ydh-i8f9j |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 221.0tok/s | no data | 166ms | r_36c-t9il61l |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 222.4tok/s | no data | 172ms | r_xbis0gb1uq3 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.6tok/s | no data | 349ms | r_rrrs_2wn0__ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.8tok/s | no data | 280ms | r_10yvku19-xt |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.3tok/s | no data | 170ms | r_sgqjiuomsug |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 219.8tok/s | no data | 179ms | r_sc_ghtmn-ti |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.8tok/s | no data | 148ms | r_p-pbvaxim4m |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 224.0tok/s | no data | 170ms | r_g1aw3mizd32 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 222.4tok/s | no data | 145ms | r_hj-6scdybyq |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 222.4tok/s | no data | 294ms | r_xsajby03vhb |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.3tok/s | no data | 154ms | r_cxuvzpfj-yn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 212.8tok/s | no data | 168ms | r_a9n_rervciy |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 217.4tok/s | no data | 199ms | r_148vtu_7e08 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 214.0tok/s | no data | 211ms | r_pnei9cgfzl5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 219.0tok/s | no data | 152ms | r_f14_uxceu0r |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.2tok/s | no data | 243ms | r_c6t0q3m_tti |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.2tok/s | no data | 191ms | r_ta1y71vmvhf |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 212.6tok/s | no data | 240ms | r_ql_2f6nzxj_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 217.5tok/s | no data | 288ms | r_0-uc27u4l1s |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 217.0tok/s | no data | 295ms | r_udgtg0c3pjv |
gemma-4-31B-it-Q4_K_M.gguf
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.37tok/s | no data | 180ms | r_uut6m_v6ui9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.27tok/s | no data | 344ms | r_23fireoga9y |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.33tok/s | no data | 156ms | r_1ofpvf4m7p5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.31tok/s | no data | 160ms | r_wwq2l7lmy6d |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.39tok/s | no data | 210ms | r_wppiqxyk9iw |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.05tok/s | no data | 207ms | r_onvh-dpmck- |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.37tok/s | no data | 176ms | r_plujbqnef08 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.36tok/s | no data | 146ms | r_n65ex7zl6ts |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.30tok/s | no data | 167ms | r_jkn4ltg393p |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.67tok/s | no data | 175ms | r_pa7nuerd9tl |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.37tok/s | no data | 157ms | r_moawmhigq5d |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.24tok/s | no data | 223ms | r_4fi6u0-u2ih |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.19tok/s | no data | 200ms | r_s0-yb8j14ys |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.39tok/s | no data | 172ms | r_n6su7ptrulh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.17tok/s | no data | 202ms | r_f1wlemwip05 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.67tok/s | no data | 209ms | r_lrxodemb-w0 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.49tok/s | no data | 221ms | r_lzv-y6_-n0b |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.97tok/s | no data | 151ms | r_0ztzrxt5qw2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.00tok/s | no data | 226ms | r_f-uojs7vckb |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.05tok/s | no data | 322ms | r_0dfevy9f4gh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.44tok/s | no data | 161ms | r_0igp5bsgx_7 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 64.71tok/s | no data | 197ms | r_608esz63ib8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.74tok/s | no data | 200ms | r_616apthaana |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.77tok/s | no data | 200ms | r_epzfs8k8ohh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.80tok/s | no data | 189ms | r_dee_mcm1ga0 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.73tok/s | no data | 228ms | r_ea3vckhh-cn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.70tok/s | no data | 156ms | r_yz3q11pa049 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.54tok/s | no data | 183ms | r_-_ag61ig4f_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.16tok/s | no data | 164ms | r_pnk9rze10h7 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.70tok/s | no data | 179ms | r__679a-4h44q |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.17tok/s | no data | 202ms | r_-dxzsj2j-y_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.75tok/s | no data | 195ms | r_wtd90zbbusq |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.19tok/s | no data | 230ms | r_q-it_4s9z4p |
Qwen3.6-27B-Q4_K_M.gguf
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.38tok/s | no data | 167ms | r_yluotk909p8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.48tok/s | no data | 172ms | r_v2vrkr4uah1 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.56tok/s | no data | 227ms | r_u4wa_y_y3vt |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.57tok/s | no data | 172ms | r_l-wg5a6o-vg |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.56tok/s | no data | 174ms | r_w482rf72v6z |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.32tok/s | no data | 290ms | r_ercjdbdw2gi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.60tok/s | no data | 382ms | r_aj808r0dw53 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.31tok/s | no data | 198ms | r_4upad7ubcpi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.00tok/s | no data | 241ms | r_qiq8q_cqfk5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.49tok/s | no data | 178ms | r_3un_x00tm0t |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.89tok/s | no data | 185ms | r_ea1db-lhv0r |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.90tok/s | no data | 165ms | r_7snxv0llk6f |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.79tok/s | no data | 297ms | r_p307g-1cdkl |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.84tok/s | no data | 398ms | r_bgylub2qqr- |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.70tok/s | no data | 167ms | r_5b-e5ortgcv |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.68tok/s | no data | 166ms | r_2g-h0epeovp |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.07tok/s | no data | 173ms | r_es8a88t0ez0 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.63tok/s | no data | 170ms | r_j0xki0x2kn2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.68tok/s | no data | 178ms | r_2673sv7x5m4 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.54tok/s | no data | 171ms | r_7u98v6--uli |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.05tok/s | no data | 191ms | r_4fxpo_ja16m |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.86tok/s | no data | 185ms | r_qhp37dyi9n1 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.56tok/s | no data | 215ms | r_5m7gbxjpw_c |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.20tok/s | no data | 291ms | r_qlo6j6lnnh3 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.95tok/s | no data | 291ms | r_q5xsw1p-7f8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.37tok/s | no data | 269ms | r_wowv__0b5s6 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.59tok/s | no data | 314ms | r_8qwzo79cf0s |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.99tok/s | no data | 317ms | r_fo83sbnxax4 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.96tok/s | no data | 190ms | r_f7ulllqu4vk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.79tok/s | no data | 187ms | r_uhv-sxt4h97 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.50tok/s | no data | 213ms | r_gipbd9a8wrn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.31tok/s | no data | 296ms | r_wax_x2ryqhk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.65tok/s | no data | 308ms | r_mqx-i-thqtq |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.90tok/s | no data | 180ms | r_jrwk2oj1el- |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.27tok/s | no data | 182ms | r_432qs45nio9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.39tok/s | no data | 225ms | r_p3m8a1lj4kg |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.56tok/s | no data | 205ms | r_qrpvh5hd9km |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.62tok/s | no data | 177ms | r_th_ounxmvpx |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.44tok/s | no data | 172ms | r_jbit-ru039h |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.04tok/s | no data | 194ms | r_wmexo75dt0o |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.26tok/s | no data | 410ms | r_-1k-8qr_v8a |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.09tok/s | no data | 241ms | r_mj85rtghwmg |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.97tok/s | no data | 189ms | r_-cpk_tabogc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.76tok/s | no data | 346ms | r_j1cr6asrnn6 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.82tok/s | no data | 413ms | r_v05axrj3dti |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.56tok/s | no data | 167ms | r_az687c4xasr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.61tok/s | no data | 183ms | r_2o9kb8hy9uk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.98tok/s | no data | 200ms | r_carzhcoe-mi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.36tok/s | no data | 168ms | r_uvy0zplf-sh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.53tok/s | no data | 188ms | r_qdm3mkehpyx |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.19tok/s | no data | 235ms | r_qlcfyt0mlv6 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.61tok/s | no data | 231ms | r_e7_858msfny |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.81tok/s | no data | 177ms | r_3hda1iz7yd8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.55tok/s | no data | 179ms | r_3n3hnvdgwnx |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.15tok/s | no data | 171ms | r_g37y03ugmpk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.48tok/s | no data | 252ms | r_p01v4lc4c11 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.45tok/s | no data | 175ms | r_lblt3fsiqzb |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.26tok/s | no data | 176ms | r_qe0el0pkw93 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.94tok/s | no data | 300ms | r_qsvrisrvp-e |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.48tok/s | no data | 288ms | r_83zbwf7c17p |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.84tok/s | no data | 350ms | r_9ap2a34-le5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.18tok/s | no data | 289ms | r_j1t3p2tae3a |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.34tok/s | no data | 301ms | r_4puowxcjcfu |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.95tok/s | no data | 329ms | r_u9eqyrsfrgt |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.23tok/s | no data | 342ms | r_q0wcxrit2bz |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.93tok/s | no data | 326ms | r_xa08qutcmkm |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.84tok/s | no data | 343ms | r_5-8ewf89dyr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.92tok/s | no data | 530ms | r_wqhqzea_7xy |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.28tok/s | no data | 553ms | r_1pww-w7p8sd |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.89tok/s | no data | 3,995ms | r_bqsunbd6xa8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 47.75tok/s | no data | 2,833ms | r_kj4fh_mmzj9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 45.92tok/s | no data | 3,089ms | r_4u7250hj28o |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 39.61tok/s | no data | 227ms | r__b89kg2iica |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.28tok/s | no data | 353ms | r_79bwm4mq_4l |
Community folklore on RTX 5090
62 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.
- communityconfidence 75%
10.00tok/s — Qwen3-Coder-Next on RTX 5090 via llama.cpp Q4_K_S
our signed data: RTX 5090 · Qwen3-Coder-Next
“95fa52dbf8ebec6acaf0105e1e9 Hey all, Just a quick one in case it saves someone else a headache. I was getting really poor throughput (\~10 tok/sec) with Qwen3-Coder-Next-Q4\_K\_S.gguf on llama.cpp, like “this can’t be right” levels, and eventually found a set of args that fix…”
- communityconfidence 75%
26.00tok/s — Qwen3-Coder-Next on RTX 5090 via llama.cpp Q4_K_S
our signed data: RTX 5090 · Qwen3-Coder-Next
“~26 tok/sec with Unsloth Qwen3-Coder-Next-Q4_K_S on RTX 5090 (Windows/llama.cpp) https://preview.redd.it/9gfytpz5srhg1.png?width=692&format=png&auto=w”
- communityconfidence 75%
207.9tok/s — Qwen3-Coder on RTX 5090 via sglang AWQ
our signed data: RTX 5090 · Qwen3-Coder
“hoosing the Framework **RTX 5090 — Qwen3-Coder-30B-A3B-Instruct-AWQ** |Metric|vLLM|SGLang| |:-|:-|:-| |Output throughput|**555.82 tok/s**|207.93 tok/s| |Mean TTFT|**549 ms**|1,558 ms| |Median TPOT|**7.06 ms**|18.84 ms| vLLM wins by 2.7x. SGLang is required `--quantization moe_…”
- communityconfidence 75%
555.8tok/s — Qwen3-Coder on RTX 5090 via sglang AWQ
our signed data: RTX 5090 · Qwen3-Coder
“atency? # 1. Choosing the Framework **RTX 5090 — Qwen3-Coder-30B-A3B-Instruct-AWQ** |Metric|vLLM|SGLang| |:-|:-|:-| |Output throughput|**555.82 tok/s**|207.93 tok/s| |Mean TTFT|**549 ms**|1,558 ms| |Median TPOT|**7.06 ms**|18.84 ms| vLLM wins by 2.7x. SGLang is required `--qu…”
- communityconfidence 70%
2.13tok/s — DeepSeek R1 on RTX 5090 via llama.cpp
our signed data: RTX 5090 · DeepSeek R1
“ok/sec *without* GPU on local gaming rig! Don't rush out and buy that 5090TI just yet (if you can even find one lol)! I just inferenced \~2.13 tok/sec with 2k context using a dynamic quant of the full R1 671B model (not a distill) after *disabling* my 3090TI GPU on a 96GB RAM g…”
- communityconfidence 70%
2.00tok/s — DeepSeek R1 on RTX 5090 via llama.cpp
our signed data: RTX 5090 · DeepSeek R1
“DeepSeek R1 671B over 2 tok/sec *without* GPU on local gaming rig! Don't rush out and buy that 5090TI just yet (if you can even find one lol)! I just inferenced \~2.13 t”
- communityconfidence 65%
196.0tok/s — on RTX 5090 via vllm AWQ
our signed data: RTX 5090
“what I learned about quant format tradeoffs on Blackwell, since I couldn’t find much written up yet. Final numbers on a single 5090: • \~196 tok/s decode • 96k context (model supports 256k native) • TTFT 1-3s warm, \~95s cold start • AWQ 4-bit (cyankiwi/gemma-4-26B-A4B-i…”
- communityconfidence 60%
150.0tok/s — GPT-OSS 120b on RTX 5090 via llama.cpp
our signed data: RTX 5090 · GPT-OSS 120b
“ir, or Devstral 24b. Here's the llama.cpp command I used to squeeze UD-Q6\_K\_XL + 48k tokens of context in my RTX 5090 VRAM and get about 150 tok/s (tg): `./llama-server --model downloaded_models/GLM-4.7-Flash-UD-Q6_K_XL.gguf --port 11433 --host "0.0.0.0" -fa on --ctx-size 480…”
- communityconfidence 60%
10.85tok/s — DeepSeek-V3.1 on RTX 5090 via llama.cpp
our signed data: RTX 5090 · DeepSeek-V3.1
“gh the system popping out through a small gap left by RTX 5090). DeepSeek-V3.1-Terminus with context = 37279 tokens: PP = 151.76 tps, TG = 10.85 tps Some things I discovered running local LLMs: * For water-cooled CPU systems, there is not enough air circulation to cool the RAM…”
- communityconfidence 60%
151.8tok/s — DeepSeek-V3.1 on RTX 5090 via llama.cpp
our signed data: RTX 5090 · DeepSeek-V3.1
“is threaded through the system popping out through a small gap left by RTX 5090). DeepSeek-V3.1-Terminus with context = 37279 tokens: PP = 151.76 tps, TG = 10.85 tps Some things I discovered running local LLMs: * For water-cooled CPU systems, there is not enough air circulatio…”
Models measured on RTX 5090
- Qwen2.5-Coder-14B-Instruct benchmarks
- Qwen3-Coder-30B-A3B-Instruct benchmarks
- gpt-oss-20b benchmarks
- Qwen3.6-27B benchmarks
- Qwen3-32B benchmarks
- Qwen3.8-27B benchmarks
- Nemotron-3.5-Bolt-74B-A9B-24GB-ctx32k benchmarks
- Nemotron-3.5-Bolt-74B-A9B-24GB benchmarks
- LFM2.5-24B-A2B benchmarks
- gemma-4-12b-it-qat benchmarks
- Coder-V2-Lite-Instruct benchmarks
- Yi-Coder-9B-Chat benchmarks
- stable-code-instruct-3b benchmarks
- Qwen2.5-32B-Instruct benchmarks
- Qwen2.5-14B-Instruct benchmarks
- Llama-3.1-8B-Instruct benchmarks
- Qwen2.5-7B-Instruct benchmarks
- phi-4 benchmarks
- gemma-2-9b-it benchmarks
- Qwen2.5-Coder-32B-Instruct benchmarks
- Qwen2.5-Coder-7B-Instruct benchmarks
- Codestral-22B-v0.1 benchmarks
- Qwen3.6-35B-A3B-Q4_K_M.gguf benchmarks
- gemma-4-31B-it-Q4_K_M.gguf benchmarks
- Qwen3.6-27B-Q4_K_M.gguf benchmarks
Common questions about RTX 5090
Direct Q&A drawn from the runs above: fastest LLM, supported model classes, backend rankings, quantization guidance.
Request a run on RTX 5090
No measured run yet for these pairs. Vote for the one you want; the most-requested runs go first. See all requests.
0 votes
We never sell or share your email. Privacy
We can rent this card · Submit your own