Qwen3.6-35B-A3B-Q4_K_M.gguf
30 workload results across 2 hardware configurations.
Fastest local config
224.0 decode tok/s
on RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB via llama.cpp. see full run
For a standard Q4_K_M quantization using llama.cpp, here are measured results for short-chat generation.
On llama.cpp Q4_K_M, Qwen3.6-35B-A3B is fastest on the RTX 5090 at 220.4 tok/s, 37% faster than the RTX 4090 (160.6 tok/s).
| Card | Decode tok/s | Run |
|---|---|---|
| RTX 5090 | 220.4 (median of 26 runs) | r_36c-t9il61l (closest to median) |
| RTX 4090 | 160.6 (single run) | r_m6j1zxwvp-f |
Qwen3.6-35B-A3B, Q4_K_M, llama.cpp, chat-short, batch 1. Only cards with their standard memory are listed; CPU-only hosts are excluded. Runs use different llama.cpp versions (RTX 5090: unknown; RTX 4090: 8931 (9725a313b)), so treat this as measured reference points, not a controlled ranking.
Local runs (30 workload results)
Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.
RTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@8931 (9725a313b) | Q4_K_M | 160.6tok/s | 464.3tok/s | 241ms | r_m6j1zxwvp-f |
| chat-long | llama.cpp@8931 (9725a313b) | Q4_K_M | 160.4tok/s | 4,588.8tok/s | 693ms | r_m6j1zxwvp-f |
| concurrent-decode | llama.cpp@8931 (9725a313b) | Q4_K_M | 163.5tok/s | no data | no data | r_m6j1zxwvp-f |
| agent-trace | llama.cpp@8931 (9725a313b) | Q4_K_M | 162.7tok/s | 6,547.5tok/s | 330ms | r_m6j1zxwvp-f |
RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp | - | 223.3tok/s | no data | 263ms | r_dp6sr2_iwcf |
| chat-short | llama.cpp | - | 224.0tok/s | no data | 167ms | r_a56-wxl21lk |
| chat-short | llama.cpp | - | 216.5tok/s | no data | 185ms | r_k070lz99uzi |
| chat-short | llama.cpp | - | 221.6tok/s | no data | 207ms | r_dqelw_a8f2c |
| chat-short | llama.cpp | - | 221.8tok/s | no data | 204ms | r_c25tn-sl4bn |
| chat-short | llama.cpp | - | 208.6tok/s | no data | 220ms | r_d1ydh-i8f9j |
| chat-short | llama.cpp | - | 221.0tok/s | no data | 166ms | r_36c-t9il61l |
| chat-short | llama.cpp | - | 222.4tok/s | no data | 172ms | r_xbis0gb1uq3 |
| chat-short | llama.cpp | - | 216.6tok/s | no data | 349ms | r_rrrs_2wn0__ |
| chat-short | llama.cpp | - | 223.8tok/s | no data | 280ms | r_10yvku19-xt |
| chat-short | llama.cpp | - | 216.3tok/s | no data | 170ms | r_sgqjiuomsug |
| chat-short | llama.cpp | - | 219.8tok/s | no data | 179ms | r_sc_ghtmn-ti |
| chat-short | llama.cpp | - | 223.8tok/s | no data | 148ms | r_p-pbvaxim4m |
| chat-short | llama.cpp | - | 224.0tok/s | no data | 170ms | r_g1aw3mizd32 |
| chat-short | llama.cpp | - | 222.4tok/s | no data | 145ms | r_hj-6scdybyq |
| chat-short | llama.cpp | - | 222.4tok/s | no data | 294ms | r_xsajby03vhb |
| chat-short | llama.cpp | - | 223.3tok/s | no data | 154ms | r_cxuvzpfj-yn |
| chat-short | llama.cpp | - | 212.8tok/s | no data | 168ms | r_a9n_rervciy |
| chat-short | llama.cpp | - | 217.4tok/s | no data | 199ms | r_148vtu_7e08 |
| chat-short | llama.cpp | - | 214.0tok/s | no data | 211ms | r_pnei9cgfzl5 |
| chat-short | llama.cpp | - | 219.0tok/s | no data | 152ms | r_f14_uxceu0r |
| chat-short | llama.cpp | - | 216.2tok/s | no data | 243ms | r_c6t0q3m_tti |
| chat-short | llama.cpp | - | 223.2tok/s | no data | 191ms | r_ta1y71vmvhf |
| chat-short | llama.cpp | - | 212.6tok/s | no data | 240ms | r_ql_2f6nzxj_ |
| chat-short | llama.cpp | - | 217.5tok/s | no data | 288ms | r_0-uc27u4l1s |
| chat-short | llama.cpp | - | 217.0tok/s | no data | 295ms | r_udgtg0c3pjv |