Leaderboard/models/qwen2-5-coder-14b-instruct

Qwen2.5-Coder-14B-Instruct-4bit

18 workload results across 5 hardware configurations.

Fastest local config

140.0 decode tok/s

on RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB via llama.cpp. see full run

For a standard Q4_K_M quantization using llama.cpp, here are measured results for short-chat generation.

On llama.cpp Q4_K_M, Qwen2.5-Coder-14B-Instruct is fastest on the RTX 5090 at 137.3 tok/s, 53% faster than the RTX 4090 (89.87 tok/s); slowest: RTX 3090 at 78.21 tok/s.

Card	Decode tok/s	TTFT (131-token prompt)	Run
RTX 5090	137.3 (single run)	65 ms	r_xnr_k4emoyr
RTX 4090	89.87 (single run)	57 ms	r_eqnsqmgz29a
RTX 3090	78.21 (single run)	77 ms	r_rkyf9wxzw9z

Qwen2.5-Coder-14B-Instruct, Q4_K_M, llama.cpp, chat-short, batch 1. Only cards with their standard memory are listed; CPU-only hosts are excluded. Runs use different llama.cpp versions (RTX 5090: 0.7.0-dev (build 1, commit 1268eda); RTX 4090: 0.6.0-dev (build 1, commit df18bee); RTX 3090: 0.7.0-dev (build 1, commit ea0c4c2)), so treat this as measured reference points, not a controlled ranking.

Local runs (18 workload results)

Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.

RTX 5090 (32GB) + AMD EPYC 7543 32-Core Processor (120c) + 945GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	137.3tok/s	2,016.9tok/s	65.0ms	r_xnr_k4emoyr
chat-long	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	128.9tok/s	5,987.2tok/s	529ms	r_xnr_k4emoyr
concurrent-decode	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	136.7tok/s	no data	no data	r_xnr_k4emoyr
agent-trace	llama.cpp@0.7.0-dev (build 1, commit 1268eda)	Q4_K_M	128.2tok/s	18,881.9tok/s	113ms	r_xnr_k4emoyr

RTX 3090 (24GB) + AMD EPYC 7282 16-Core Processor (16c) + 126GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	78.21tok/s	1,693.2tok/s	77.4ms	r_rkyf9wxzw9z
chat-long	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	74.31tok/s	2,549.2tok/s	1,243ms	r_rkyf9wxzw9z
concurrent-decode	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	77.11tok/s	no data	no data	r_rkyf9wxzw9z
agent-trace	llama.cpp@0.7.0-dev (build 1, commit ea0c4c2)	Q4_K_M	75.86tok/s	7,427.0tok/s	273ms	r_rkyf9wxzw9z

RTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp@0.6.0-dev (build 1, commit df18bee)	Q4_K_M	89.87tok/s	2,292.6tok/s	57.1ms	r_eqnsqmgz29a
chat-long	llama.cpp@0.6.0-dev (build 1, commit df18bee)	Q4_K_M	84.17tok/s	4,989.1tok/s	635ms	r_eqnsqmgz29a
concurrent-decode	llama.cpp@0.6.0-dev (build 1, commit df18bee)	Q4_K_M	88.90tok/s	no data	no data	r_eqnsqmgz29a
agent-trace	llama.cpp@0.6.0-dev (build 1, commit df18bee)	Q4_K_M	86.64tok/s	14,684.0tok/s	142ms	r_eqnsqmgz29a

RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	llama.cpp	-	140.0tok/s	no data	42.1ms	r_opuj21f13-_
chat-long	llama.cpp	-	123.0tok/s	no data	457ms	r_opuj21f13-_
concurrent-decode	llama.cpp	-	133.8tok/s	no data	no data	r_opuj21f13-_
agent-trace	llama.cpp	-	130.5tok/s	21,985.9tok/s	104ms	r_opuj21f13-_
chat-short	llama.cpp	-	136.0tok/s	no data	51.3ms	r_p_f63tcgans

M3 Ultra (60-core GPU) + 96GB unified

Workload	Backend	Quant	decode tok/s	prefill tok/s	TTFT	Run
chat-short	mlx@0.31.3	-	70.51tok/s	306.1tok/s	428ms	r_l36cijqxq4t