Qwen3.6-27B-Q4_K_M.gguf

86 workload results across 4 hardware configurations.

Fastest local config

75.1 decode tok/s

on RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB via llama.cpp (Q4_K_M). see full run

For a standard Q4_K_M quantization using llama.cpp, here are measured results for short-chat generation.

On llama.cpp Q4_K_M, Qwen3.6-27B is fastest on the RTX 5090 at 70.89 tok/s, 50% faster than the RTX 4090 (47.14 tok/s); slowest: RTX 3090 at 42.73 tok/s.

Card	Decode tok/s	Run
RTX 5090	70.89 (median of 75 runs)	r_ea1db-lhv0r (closest to median)
RTX 4090	47.14 (single run)	r_d_wda_rw62f
RTX 3090	42.73 (single run)	r_qm1xyvjmx_r

Qwen3.6-27B, Q4_K_M, llama.cpp, chat-short, batch 1. Only cards with their standard memory are listed; CPU-only hosts are excluded. Runs use different llama.cpp versions (RTX 5090: unknown; RTX 4090: 0.6.0-dev (build 1, commit 3fb4767); RTX 3090: 0.7.0-dev (build 1, commit ea0c4c2)), so treat this as measured reference points, not a controlled ranking.

Local runs (86 workload results)

Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.

RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	75.09tok/s	302.2tok/s	371ms	r_pob-co51bd-
chat-long	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	74.55tok/s	2,644.8tok/s	1,203ms	r_pob-co51bd-
concurrent-decode	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	75.13tok/s	no data	no data	r_pob-co51bd-
agent-trace	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	74.13tok/s	4,464.5tok/s	485ms	r_pob-co51bd-

RTX 3090 (24GB) + AMD EPYC 7282 16-Core Processor (16c) + 126GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	42.73tok/s	236.9tok/s	473ms	r_qm1xyvjmx_r
chat-long	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	42.26tok/s	1,321.5tok/s	2,408ms	r_qm1xyvjmx_r
concurrent-decode	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	42.63tok/s	no data	no data	r_qm1xyvjmx_r
agent-trace	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	42.07tok/s	2,369.6tok/s	881ms	r_qm1xyvjmx_r

RTX 4090 (24GB) + AMD EPYC 7763 64-Core Processor (128c) + 1008GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp@0.6.0-dev (build 1, commit 3fb4767)	Q4_K_M	47.14tok/s	349.5tok/s	320ms	r_d_wda_rw62f
chat-long	llama.cpp@0.6.0-dev (build 1, commit 3fb4767)	Q4_K_M	46.57tok/s	2,107.9tok/s	1,510ms	r_d_wda_rw62f
concurrent-decode	llama.cpp@0.6.0-dev (build 1, commit 3fb4767)	Q4_K_M	47.15tok/s	no data	no data	r_d_wda_rw62f
agent-trace	llama.cpp@0.6.0-dev (build 1, commit 3fb4767)	Q4_K_M	46.19tok/s	3,445.6tok/s	626ms	r_d_wda_rw62f

RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp	-	72.38tok/s	no data	167ms	r_yluotk909p8
chat-short	llama.cpp	-	72.48tok/s	no data	172ms	r_v2vrkr4uah1
chat-short	llama.cpp	-	69.56tok/s	no data	227ms	r_u4wa_y_y3vt
chat-short	llama.cpp	-	72.57tok/s	no data	172ms	r_l-wg5a6o-vg
chat-short	llama.cpp	-	69.56tok/s	no data	174ms	r_w482rf72v6z
chat-short	llama.cpp	-	72.32tok/s	no data	290ms	r_ercjdbdw2gi
chat-short	llama.cpp	-	69.60tok/s	no data	382ms	r_aj808r0dw53
chat-short	llama.cpp	-	72.31tok/s	no data	198ms	r_4upad7ubcpi
chat-short	llama.cpp	-	68.00tok/s	no data	241ms	r_qiq8q_cqfk5
chat-short	llama.cpp	-	72.49tok/s	no data	178ms	r_3un_x00tm0t
chat-short	llama.cpp	-	70.89tok/s	no data	185ms	r_ea1db-lhv0r
chat-short	llama.cpp	-	72.90tok/s	no data	165ms	r_7snxv0llk6f
chat-short	llama.cpp	-	72.79tok/s	no data	297ms	r_p307g-1cdkl
chat-short	llama.cpp	-	69.84tok/s	no data	398ms	r_bgylub2qqr-
chat-short	llama.cpp	-	69.70tok/s	no data	167ms	r_5b-e5ortgcv
chat-short	llama.cpp	-	72.68tok/s	no data	166ms	r_2g-h0epeovp
chat-short	llama.cpp	-	70.07tok/s	no data	173ms	r_es8a88t0ez0
chat-short	llama.cpp	-	72.63tok/s	no data	170ms	r_j0xki0x2kn2
chat-short	llama.cpp	-	69.68tok/s	no data	178ms	r_2673sv7x5m4
chat-short	llama.cpp	-	72.54tok/s	no data	171ms	r_7u98v6--uli
chat-short	llama.cpp	-	69.05tok/s	no data	191ms	r_4fxpo_ja16m
chat-short	llama.cpp	-	67.86tok/s	no data	185ms	r_qhp37dyi9n1
chat-short	llama.cpp	-	72.56tok/s	no data	215ms	r_5m7gbxjpw_c
chat-short	llama.cpp	-	73.20tok/s	no data	291ms	r_qlo6j6lnnh3
chat-short	llama.cpp	-	69.95tok/s	no data	291ms	r_q5xsw1p-7f8
chat-short	llama.cpp	-	70.37tok/s	no data	269ms	r_wowv__0b5s6
chat-short	llama.cpp	-	72.59tok/s	no data	314ms	r_8qwzo79cf0s
chat-short	llama.cpp	-	71.99tok/s	no data	317ms	r_fo83sbnxax4
chat-short	llama.cpp	-	73.96tok/s	no data	190ms	r_f7ulllqu4vk
chat-short	llama.cpp	-	72.79tok/s	no data	187ms	r_uhv-sxt4h97
chat-short	llama.cpp	-	71.50tok/s	no data	213ms	r_gipbd9a8wrn
chat-short	llama.cpp	-	73.31tok/s	no data	296ms	r_wax_x2ryqhk
chat-short	llama.cpp	-	69.65tok/s	no data	308ms	r_mqx-i-thqtq
chat-short	llama.cpp	-	69.90tok/s	no data	180ms	r_jrwk2oj1el-
chat-short	llama.cpp	-	72.27tok/s	no data	182ms	r_432qs45nio9
chat-short	llama.cpp	-	72.39tok/s	no data	225ms	r_p3m8a1lj4kg
chat-short	llama.cpp	-	69.56tok/s	no data	205ms	r_qrpvh5hd9km
chat-short	llama.cpp	-	69.62tok/s	no data	177ms	r_th_ounxmvpx
chat-short	llama.cpp	-	72.44tok/s	no data	172ms	r_jbit-ru039h
chat-short	llama.cpp	-	70.04tok/s	no data	194ms	r_wmexo75dt0o
chat-short	llama.cpp	-	71.26tok/s	no data	410ms	r_-1k-8qr_v8a
chat-short	llama.cpp	-	71.09tok/s	no data	241ms	r_mj85rtghwmg
chat-short	llama.cpp	-	68.97tok/s	no data	189ms	r_-cpk_tabogc
chat-short	llama.cpp	-	73.76tok/s	no data	346ms	r_j1cr6asrnn6
chat-short	llama.cpp	-	68.82tok/s	no data	413ms	r_v05axrj3dti
chat-short	llama.cpp	-	73.56tok/s	no data	167ms	r_az687c4xasr
chat-short	llama.cpp	-	68.61tok/s	no data	183ms	r_2o9kb8hy9uk
chat-short	llama.cpp	-	70.98tok/s	no data	200ms	r_carzhcoe-mi
chat-short	llama.cpp	-	72.36tok/s	no data	168ms	r_uvy0zplf-sh
chat-short	llama.cpp	-	69.53tok/s	no data	188ms	r_qdm3mkehpyx
chat-short	llama.cpp	-	72.19tok/s	no data	235ms	r_qlcfyt0mlv6
chat-short	llama.cpp	-	69.61tok/s	no data	231ms	r_e7_858msfny
chat-short	llama.cpp	-	71.81tok/s	no data	177ms	r_3hda1iz7yd8
chat-short	llama.cpp	-	69.55tok/s	no data	179ms	r_3n3hnvdgwnx
chat-short	llama.cpp	-	72.15tok/s	no data	171ms	r_g37y03ugmpk
chat-short	llama.cpp	-	69.48tok/s	no data	252ms	r_p01v4lc4c11
chat-short	llama.cpp	-	69.45tok/s	no data	175ms	r_lblt3fsiqzb
chat-short	llama.cpp	-	73.26tok/s	no data	176ms	r_qe0el0pkw93
chat-short	llama.cpp	-	68.94tok/s	no data	300ms	r_qsvrisrvp-e
chat-short	llama.cpp	-	72.48tok/s	no data	288ms	r_83zbwf7c17p
chat-short	llama.cpp	-	69.84tok/s	no data	350ms	r_9ap2a34-le5
chat-short	llama.cpp	-	72.18tok/s	no data	289ms	r_j1t3p2tae3a
chat-short	llama.cpp	-	69.34tok/s	no data	301ms	r_4puowxcjcfu
chat-short	llama.cpp	-	70.95tok/s	no data	329ms	r_u9eqyrsfrgt
chat-short	llama.cpp	-	69.23tok/s	no data	342ms	r_q0wcxrit2bz
chat-short	llama.cpp	-	68.93tok/s	no data	326ms	r_xa08qutcmkm
chat-short	llama.cpp	-	69.84tok/s	no data	343ms	r_5-8ewf89dyr
chat-short	llama.cpp	-	71.92tok/s	no data	530ms	r_wqhqzea_7xy
chat-short	llama.cpp	-	67.28tok/s	no data	553ms	r_1pww-w7p8sd
chat-short	llama.cpp	-	69.89tok/s	no data	3,995ms	r_bqsunbd6xa8
chat-short	llama.cpp	-	47.75tok/s	no data	2,833ms	r_kj4fh_mmzj9
chat-short	llama.cpp	-	45.92tok/s	no data	3,089ms	r_4u7250hj28o
chat-short	llama.cpp	-	39.61tok/s	no data	227ms	r__b89kg2iica
chat-short	llama.cpp	-	66.28tok/s	no data	353ms	r_79bwm4mq_4l

RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GBRTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB

RTX 3090 (24GB) + AMD EPYC 7282 16-Core Processor (16c) + 126GBRTX 3090 (24GB) + AMD EPYC 7282 16-Core Processor (16c) + 126GB

RTX 4090 (24GB) + AMD EPYC 7763 64-Core Processor (128c) + 1008GBRTX 4090 (24GB) + AMD EPYC 7763 64-Core Processor (128c) + 1008GB

RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GBRTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB