RTX 5090 LLM benchmark
On the RTX 5090, measured single-user chat decode speeds are: ~7–9B models at 229.4 tok/s (llama.cpp; Measured, middle of 10 results, source run), ~12–14B models at 137.3 tok/s (Q4_0, llama.cpp; Measured, middle of 10 results, source run) and ~22–32B models at 69.6 tok/s (llama.cpp; Measured, middle of 121 results, source run). Short-chat workload, local runtimes, dense models only; quantization, backend and settings vary between submissions, so check each row in the tables below.
Fastest single result in these submissions (any model size)
341.9 decode tok/s
stable-code-instruct-3b via llama.cpp. Workload: chat-short. Reported hardware: RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB. see full run
Qwen2.5-Coder-14B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 137.3tok/s | 2,016.9tok/s | 65.0ms | r_xnr_k4emoyr |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 128.9tok/s | 5,987.2tok/s | 529ms | r_xnr_k4emoyr |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 136.7tok/s | no data | no data | r_xnr_k4emoyr |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 128.2tok/s | 18,881.9tok/s | 113ms | r_xnr_k4emoyr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 140.0tok/s | no data | 42.1ms | r_opuj21f13-_ |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 123.0tok/s | no data | 457ms | r_opuj21f13-_ |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 133.8tok/s | no data | no data | r_opuj21f13-_ |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 130.5tok/s | 21,985.9tok/s | 104ms | r_opuj21f13-_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 136.0tok/s | no data | 51.3ms | r_p_f63tcgans |
Qwen3-Coder-30B-A3B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 304.9tok/s | 514.5tok/s | 214ms | r_vsype4pm1ko |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 266.2tok/s | 8,353.2tok/s | 377ms | r_vsype4pm1ko |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 296.7tok/s | no data | no data | r_vsype4pm1ko |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 278.8tok/s | 23,318.7tok/s | 96.6ms | r_vsype4pm1ko |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.41tok/s | no data | 334ms | r_pm_a1uf2ufc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.55tok/s | no data | 1,204ms | r_pm_a1uf2ufc |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.04tok/s | no data | no data | r_pm_a1uf2ufc |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.09tok/s | 4,816.7tok/s | 451ms | r_pm_a1uf2ufc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 259.9tok/s | no data | 218ms | r_c7qyvvmmsv1 |
gpt-oss-20b
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 324.2tok/s | 586.8tok/s | 283ms | r_dwp5fgdapmw |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 311.0tok/s | 12,638.4tok/s | 252ms | r_dwp5fgdapmw |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 322.5tok/s | no data | no data | r_dwp5fgdapmw |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 319.6tok/s | 23,887.0tok/s | 89.2ms | r_dwp5fgdapmw |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 318.4tok/s | no data | 173ms | r_b9ul-vxh9sc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 301.6tok/s | no data | 239ms | r_b9ul-vxh9sc |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 302.8tok/s | no data | no data | r_b9ul-vxh9sc |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 304.5tok/s | 26,907.5tok/s | 77.7ms | r_b9ul-vxh9sc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.42tok/s | no data | 333ms | r_r9h57uts9lr |
Qwen3.6-27B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 75.09tok/s | 302.2tok/s | 371ms | r_pob-co51bd- |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 74.55tok/s | 2,644.8tok/s | 1,203ms | r_pob-co51bd- |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 75.13tok/s | no data | no data | r_pob-co51bd- |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 74.13tok/s | 4,464.5tok/s | 485ms | r_pob-co51bd- |
Qwen3-32B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 69.83tok/s | 1,047.0tok/s | 105ms | r_vhbqyt1f_xr |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 64.13tok/s | 3,139.6tok/s | 1,002ms | r_vhbqyt1f_xr |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 68.52tok/s | no data | no data | r_vhbqyt1f_xr |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB | llama.cpp@0.7.0-dev (build 1, commit 1268eda) | Q4_K_M | 66.46tok/s | 8,161.5tok/s | 253ms | r_vhbqyt1f_xr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.51tok/s | no data | 158ms | r_-txe_hiq44n |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 62.30tok/s | no data | 1,067ms | r_-txe_hiq44n |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.64tok/s | no data | no data | r_-txe_hiq44n |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 64.34tok/s | 8,394.1tok/s | 259ms | r_-txe_hiq44n |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.45tok/s | no data | 327ms | r_phvxm9dcak0 |
Qwen3.8-27B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 75.19tok/s | 563.4tok/s | 273ms | r_qfrb2zmyk0k |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 74.57tok/s | 2,499.5tok/s | 1,290ms | r_qfrb2zmyk0k |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 75.40tok/s | no data | no data | r_qfrb2zmyk0k |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 73.52tok/s | 5,065.3tok/s | 436ms | r_qfrb2zmyk0k |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.66tok/s | 969.8tok/s | 118ms | r_dz5a3lzxhrj |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.10tok/s | 3,496.8tok/s | 911ms | r_dz5a3lzxhrj |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.73tok/s | 973.6tok/s | 117ms | r_c3ps9wjygi9 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.09tok/s | 3,506.5tok/s | 908ms | r_c3ps9wjygi9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.43tok/s | 865.3tok/s | 132ms | r_7mod38qsldj |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_K_M | 65.10tok/s | 3,501.1tok/s | 909ms | r_7mod38qsldj |
Nemotron-3.5-Bolt-74B-A9B-24GB-ctx32k
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 157.3tok/s | 355.4tok/s | 338ms | r_ymx58emedxd |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 155.7tok/s | 2,605.8tok/s | 1,247ms | r_ymx58emedxd |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 157.7tok/s | no data | no data | r_ymx58emedxd |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 154.9tok/s | 5,095.7tok/s | 431ms | r_ymx58emedxd |
Nemotron-3.5-Bolt-74B-A9B-24GB
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 154.7tok/s | 353.4tok/s | 340ms | r_z-l1yelv_67 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 154.2tok/s | 2,623.3tok/s | 1,239ms | r_z-l1yelv_67 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 156.0tok/s | no data | no data | r_z-l1yelv_67 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | - | 149.3tok/s | 5,097.2tok/s | 418ms | r_z-l1yelv_67 |
LFM2.5-24B-A2B
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 308.5tok/s | 553.1tok/s | 206ms | r_c8k6fleym09 |
| chat-long | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 300.4tok/s | 12,418.3tok/s | 264ms | r_c8k6fleym09 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 321.3tok/s | no data | no data | r_c8k6fleym09 |
| agent-trace | not supplied | RTX 5090 (32GB) + INTEL(R) XEON(R) GOLD 6530 (64c) + 1008GB | llama.cpp@0.6.0-dev (build 1, commit a0e6dc7) | Q4_K_M | 313.2tok/s | 18,108.8tok/s | 117ms | r_c8k6fleym09 |
gemma-4-12b-it-qat
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| long-context-decay | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 142.6tok/s | 7,115.8tok/s | 3,355ms | r_-lh-vczf7ik |
| long-context-decay | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 136.3tok/s | 7,161.0tok/s | 3,334ms | r_1srqmpjnkgd |
| long-context-decay | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 134.2tok/s | 6,022.4tok/s | 3,964ms | r_8q-uf-rq-gp |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 141.1tok/s | 2,027.9tok/s | 59.7ms | r_v5arqmazf31 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 135.6tok/s | 5,147.7tok/s | 619ms | r_v5arqmazf31 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 139.5tok/s | 1,958.9tok/s | 61.8ms | r_c2podhe9qwc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 137.1tok/s | 5,118.0tok/s | 623ms | r_c2podhe9qwc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 137.3tok/s | 2,316.4tok/s | 52.2ms | r_8xmm65n1fhq |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp@1 (9725a31) | Q4_0 | 135.6tok/s | 5,523.3tok/s | 577ms | r_8xmm65n1fhq |
Coder-V2-Lite-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 293.1tok/s | no data | 297ms | r_bfpto9so2o1 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 181.2tok/s | no data | 361ms | r_bfpto9so2o1 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 268.4tok/s | no data | no data | r_bfpto9so2o1 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 204.5tok/s | 24,978.7tok/s | 80.7ms | r_bfpto9so2o1 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 309.5tok/s | no data | 268ms | r_0_gs1rgl2fl |
Yi-Coder-9B-Chat
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.79tok/s | no data | 317ms | r_zlh6az5q0o_ |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.60tok/s | no data | 1,195ms | r_zlh6az5q0o_ |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.99tok/s | no data | no data | r_zlh6az5q0o_ |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.76tok/s | 4,478.0tok/s | 397ms | r_zlh6az5q0o_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 199.3tok/s | no data | 36.2ms | r_u4iojm6-ekg |
stable-code-instruct-3b
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 341.9tok/s | no data | 175ms | r_q9f15lz6831 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 289.1tok/s | no data | 232ms | r_q9f15lz6831 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 356.1tok/s | no data | no data | r_q9f15lz6831 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 292.8tok/s | 44,387.5tok/s | 42.7ms | r_q9f15lz6831 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 331.0tok/s | no data | 36.7ms | r_8l57cim1i10 |
Qwen2.5-32B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.71tok/s | no data | 77.0ms | r_bjy5a5izxjc |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 65.36tok/s | no data | 968ms | r_bjy5a5izxjc |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.60tok/s | no data | no data | r_bjy5a5izxjc |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.45tok/s | 10,407.9tok/s | 214ms | r_bjy5a5izxjc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.90tok/s | no data | 115ms | r_twfs86tf_xf |
Qwen2.5-14B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 130.3tok/s | no data | 48.6ms | r_xr4qdv1hgf2 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 122.7tok/s | no data | 489ms | r_xr4qdv1hgf2 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 131.4tok/s | no data | no data | r_xr4qdv1hgf2 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 122.8tok/s | 20,315.4tok/s | 111ms | r_xr4qdv1hgf2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 133.3tok/s | no data | 55.9ms | r_tj9bu7gvnvh |
Llama-3.1-8B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 229.4tok/s | no data | 35.6ms | r_qr4srge34da |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 211.4tok/s | no data | 266ms | r_qr4srge34da |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.0tok/s | no data | no data | r_qr4srge34da |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 210.3tok/s | 36,720.3tok/s | 60.8ms | r_qr4srge34da |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.2tok/s | no data | 33.8ms | r_kfrkg-vn376 |
Qwen2.5-7B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.7tok/s | no data | 37.0ms | r_1shiviswt3d |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.4tok/s | no data | 239ms | r_1shiviswt3d |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 240.7tok/s | no data | no data | r_1shiviswt3d |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 232.5tok/s | 38,007.7tok/s | 57.1ms | r_1shiviswt3d |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 246.7tok/s | no data | 33.9ms | r_3yn-4321hp- |
phi-4
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 139.1tok/s | no data | 51.1ms | r_k1u_k1j_1i2 |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 133.7tok/s | no data | 433ms | r_k1u_k1j_1i2 |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 140.2tok/s | no data | no data | r_k1u_k1j_1i2 |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 124.8tok/s | 21,162.4tok/s | 102ms | r_k1u_k1j_1i2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 140.9tok/s | no data | 56.7ms | r_e-k4aea8ipr |
gemma-2-9b-it
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.45tok/s | no data | 325ms | r_1_xl4zb5-xj |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.42tok/s | no data | 1,044ms | r_1_xl4zb5-xj |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.57tok/s | no data | no data | r_1_xl4zb5-xj |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.46tok/s | 4,816.8tok/s | 420ms | r_1_xl4zb5-xj |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 152.7tok/s | no data | 58.9ms | r__b_bzmmab_8 |
Qwen2.5-Coder-32B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.41tok/s | no data | 384ms | r_v983y0y3r2u |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.15tok/s | no data | 1,251ms | r_v983y0y3r2u |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.96tok/s | no data | no data | r_v983y0y3r2u |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.56tok/s | 4,677.1tok/s | 414ms | r_v983y0y3r2u |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.91tok/s | no data | 71.0ms | r_nkbs6d3-d21 |
Qwen2.5-Coder-7B-Instruct
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 255.3tok/s | no data | 35.4ms | r_2b9o6y_49mi |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 244.0tok/s | no data | 228ms | r_2b9o6y_49mi |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 254.0tok/s | no data | no data | r_2b9o6y_49mi |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 229.9tok/s | 38,735.2tok/s | 57.4ms | r_2b9o6y_49mi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 244.9tok/s | no data | 36.1ms | r_mln72x5zbis |
Codestral-22B-v0.1
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 93.72tok/s | no data | 68.6ms | r_4q040m4scic |
| chat-long | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 87.36tok/s | no data | 849ms | r_4q040m4scic |
| concurrent-decode | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 100.3tok/s | no data | no data | r_4q040m4scic |
| agent-trace | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 93.44tok/s | 10,351.0tok/s | 191ms | r_4q040m4scic |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.95tok/s | no data | 199ms | r_sqr8liqh4ii |
Qwen3.6-35B-A3B-Q4_K_M.gguf
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.3tok/s | no data | 263ms | r_dp6sr2_iwcf |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 224.0tok/s | no data | 167ms | r_a56-wxl21lk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.5tok/s | no data | 185ms | r_k070lz99uzi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 221.6tok/s | no data | 207ms | r_dqelw_a8f2c |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 221.8tok/s | no data | 204ms | r_c25tn-sl4bn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 208.6tok/s | no data | 220ms | r_d1ydh-i8f9j |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 221.0tok/s | no data | 166ms | r_36c-t9il61l |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 222.4tok/s | no data | 172ms | r_xbis0gb1uq3 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.6tok/s | no data | 349ms | r_rrrs_2wn0__ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.8tok/s | no data | 280ms | r_10yvku19-xt |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.3tok/s | no data | 170ms | r_sgqjiuomsug |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 219.8tok/s | no data | 179ms | r_sc_ghtmn-ti |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.8tok/s | no data | 148ms | r_p-pbvaxim4m |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 224.0tok/s | no data | 170ms | r_g1aw3mizd32 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 222.4tok/s | no data | 145ms | r_hj-6scdybyq |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 222.4tok/s | no data | 294ms | r_xsajby03vhb |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.3tok/s | no data | 154ms | r_cxuvzpfj-yn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 212.8tok/s | no data | 168ms | r_a9n_rervciy |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 217.4tok/s | no data | 199ms | r_148vtu_7e08 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 214.0tok/s | no data | 211ms | r_pnei9cgfzl5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 219.0tok/s | no data | 152ms | r_f14_uxceu0r |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 216.2tok/s | no data | 243ms | r_c6t0q3m_tti |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 223.2tok/s | no data | 191ms | r_ta1y71vmvhf |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 212.6tok/s | no data | 240ms | r_ql_2f6nzxj_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 217.5tok/s | no data | 288ms | r_0-uc27u4l1s |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 217.0tok/s | no data | 295ms | r_udgtg0c3pjv |
gemma-4-31B-it-Q4_K_M.gguf
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.37tok/s | no data | 180ms | r_uut6m_v6ui9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.27tok/s | no data | 344ms | r_23fireoga9y |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.33tok/s | no data | 156ms | r_1ofpvf4m7p5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.31tok/s | no data | 160ms | r_wwq2l7lmy6d |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.39tok/s | no data | 210ms | r_wppiqxyk9iw |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.05tok/s | no data | 207ms | r_onvh-dpmck- |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.37tok/s | no data | 176ms | r_plujbqnef08 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.36tok/s | no data | 146ms | r_n65ex7zl6ts |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.30tok/s | no data | 167ms | r_jkn4ltg393p |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.67tok/s | no data | 175ms | r_pa7nuerd9tl |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.37tok/s | no data | 157ms | r_moawmhigq5d |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.24tok/s | no data | 223ms | r_4fi6u0-u2ih |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.19tok/s | no data | 200ms | r_s0-yb8j14ys |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.39tok/s | no data | 172ms | r_n6su7ptrulh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.17tok/s | no data | 202ms | r_f1wlemwip05 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.67tok/s | no data | 209ms | r_lrxodemb-w0 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.49tok/s | no data | 221ms | r_lzv-y6_-n0b |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.97tok/s | no data | 151ms | r_0ztzrxt5qw2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.00tok/s | no data | 226ms | r_f-uojs7vckb |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.05tok/s | no data | 322ms | r_0dfevy9f4gh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.44tok/s | no data | 161ms | r_0igp5bsgx_7 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 64.71tok/s | no data | 197ms | r_608esz63ib8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.74tok/s | no data | 200ms | r_616apthaana |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.77tok/s | no data | 200ms | r_epzfs8k8ohh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.80tok/s | no data | 189ms | r_dee_mcm1ga0 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.73tok/s | no data | 228ms | r_ea3vckhh-cn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.70tok/s | no data | 156ms | r_yz3q11pa049 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.54tok/s | no data | 183ms | r_-_ag61ig4f_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.16tok/s | no data | 164ms | r_pnk9rze10h7 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.70tok/s | no data | 179ms | r__679a-4h44q |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.17tok/s | no data | 202ms | r_-dxzsj2j-y_ |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.75tok/s | no data | 195ms | r_wtd90zbbusq |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.19tok/s | no data | 230ms | r_q-it_4s9z4p |
Qwen3.6-27B-Q4_K_M.gguf
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.38tok/s | no data | 167ms | r_yluotk909p8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.48tok/s | no data | 172ms | r_v2vrkr4uah1 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.56tok/s | no data | 227ms | r_u4wa_y_y3vt |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.57tok/s | no data | 172ms | r_l-wg5a6o-vg |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.56tok/s | no data | 174ms | r_w482rf72v6z |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.32tok/s | no data | 290ms | r_ercjdbdw2gi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.60tok/s | no data | 382ms | r_aj808r0dw53 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.31tok/s | no data | 198ms | r_4upad7ubcpi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.00tok/s | no data | 241ms | r_qiq8q_cqfk5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.49tok/s | no data | 178ms | r_3un_x00tm0t |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.89tok/s | no data | 185ms | r_ea1db-lhv0r |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.90tok/s | no data | 165ms | r_7snxv0llk6f |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.79tok/s | no data | 297ms | r_p307g-1cdkl |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.84tok/s | no data | 398ms | r_bgylub2qqr- |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.70tok/s | no data | 167ms | r_5b-e5ortgcv |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.68tok/s | no data | 166ms | r_2g-h0epeovp |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.07tok/s | no data | 173ms | r_es8a88t0ez0 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.63tok/s | no data | 170ms | r_j0xki0x2kn2 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.68tok/s | no data | 178ms | r_2673sv7x5m4 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.54tok/s | no data | 171ms | r_7u98v6--uli |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.05tok/s | no data | 191ms | r_4fxpo_ja16m |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.86tok/s | no data | 185ms | r_qhp37dyi9n1 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.56tok/s | no data | 215ms | r_5m7gbxjpw_c |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.20tok/s | no data | 291ms | r_qlo6j6lnnh3 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.95tok/s | no data | 291ms | r_q5xsw1p-7f8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.37tok/s | no data | 269ms | r_wowv__0b5s6 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.59tok/s | no data | 314ms | r_8qwzo79cf0s |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.99tok/s | no data | 317ms | r_fo83sbnxax4 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.96tok/s | no data | 190ms | r_f7ulllqu4vk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.79tok/s | no data | 187ms | r_uhv-sxt4h97 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.50tok/s | no data | 213ms | r_gipbd9a8wrn |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.31tok/s | no data | 296ms | r_wax_x2ryqhk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.65tok/s | no data | 308ms | r_mqx-i-thqtq |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.90tok/s | no data | 180ms | r_jrwk2oj1el- |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.27tok/s | no data | 182ms | r_432qs45nio9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.39tok/s | no data | 225ms | r_p3m8a1lj4kg |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.56tok/s | no data | 205ms | r_qrpvh5hd9km |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.62tok/s | no data | 177ms | r_th_ounxmvpx |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.44tok/s | no data | 172ms | r_jbit-ru039h |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.04tok/s | no data | 194ms | r_wmexo75dt0o |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.26tok/s | no data | 410ms | r_-1k-8qr_v8a |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.09tok/s | no data | 241ms | r_mj85rtghwmg |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.97tok/s | no data | 189ms | r_-cpk_tabogc |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.76tok/s | no data | 346ms | r_j1cr6asrnn6 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.82tok/s | no data | 413ms | r_v05axrj3dti |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.56tok/s | no data | 167ms | r_az687c4xasr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.61tok/s | no data | 183ms | r_2o9kb8hy9uk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.98tok/s | no data | 200ms | r_carzhcoe-mi |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.36tok/s | no data | 168ms | r_uvy0zplf-sh |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.53tok/s | no data | 188ms | r_qdm3mkehpyx |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.19tok/s | no data | 235ms | r_qlcfyt0mlv6 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.61tok/s | no data | 231ms | r_e7_858msfny |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.81tok/s | no data | 177ms | r_3hda1iz7yd8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.55tok/s | no data | 179ms | r_3n3hnvdgwnx |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.15tok/s | no data | 171ms | r_g37y03ugmpk |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.48tok/s | no data | 252ms | r_p01v4lc4c11 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.45tok/s | no data | 175ms | r_lblt3fsiqzb |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 73.26tok/s | no data | 176ms | r_qe0el0pkw93 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.94tok/s | no data | 300ms | r_qsvrisrvp-e |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.48tok/s | no data | 288ms | r_83zbwf7c17p |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.84tok/s | no data | 350ms | r_9ap2a34-le5 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 72.18tok/s | no data | 289ms | r_j1t3p2tae3a |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.34tok/s | no data | 301ms | r_4puowxcjcfu |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 70.95tok/s | no data | 329ms | r_u9eqyrsfrgt |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.23tok/s | no data | 342ms | r_q0wcxrit2bz |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 68.93tok/s | no data | 326ms | r_xa08qutcmkm |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.84tok/s | no data | 343ms | r_5-8ewf89dyr |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 71.92tok/s | no data | 530ms | r_wqhqzea_7xy |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 67.28tok/s | no data | 553ms | r_1pww-w7p8sd |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 69.89tok/s | no data | 3,995ms | r_bqsunbd6xa8 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 47.75tok/s | no data | 2,833ms | r_kj4fh_mmzj9 |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 45.92tok/s | no data | 3,089ms | r_4u7250hj28o |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 39.61tok/s | no data | 227ms | r__b89kg2iica |
| chat-short | not supplied | RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB | llama.cpp | - | 66.28tok/s | no data | 353ms | r_79bwm4mq_4l |
Models measured on RTX 5090
- Qwen2.5-Coder-14B-Instruct benchmarks
- Qwen3-Coder-30B-A3B-Instruct benchmarks
- gpt-oss-20b benchmarks
- Qwen3.6-27B benchmarks
- Qwen3-32B benchmarks
- Qwen3.8-27B benchmarks
- Nemotron-3.5-Bolt-74B-A9B-24GB-ctx32k benchmarks
- Nemotron-3.5-Bolt-74B-A9B-24GB benchmarks
- LFM2.5-24B-A2B benchmarks
- gemma-4-12b-it-qat benchmarks
- Coder-V2-Lite-Instruct benchmarks
- Yi-Coder-9B-Chat benchmarks
- stable-code-instruct-3b benchmarks
- Qwen2.5-32B-Instruct benchmarks
- Qwen2.5-14B-Instruct benchmarks
- Llama-3.1-8B-Instruct benchmarks
- Qwen2.5-7B-Instruct benchmarks
- phi-4 benchmarks
- gemma-2-9b-it benchmarks
- Qwen2.5-Coder-32B-Instruct benchmarks
- Qwen2.5-Coder-7B-Instruct benchmarks
- Codestral-22B-v0.1 benchmarks
- Qwen3.6-35B-A3B-Q4_K_M.gguf benchmarks
- gemma-4-31B-it-Q4_K_M.gguf benchmarks
- Qwen3.6-27B-Q4_K_M.gguf benchmarks