Qwen2.5-7B-Instruct vs Llama-3.1-8B-Instruct
MeasuredOn the same M3 Ultra (same quant and runtime family, chat-short), Qwen2.5-7B-Instruct decoded at 139.6 tok/s and Llama-3.1-8B-Instruct at 130.2 tok/s, each model's fastest matched run on that hardware.
Signed, community-submitted decode tok/s, prefill, and TTFT for Qwen2.5-7B-Instruct vs Llama-3.1-8B-Instruct — every number links to the run it came from.
How to read these results
On the same M3 Ultra, Qwen2.5-7B-Instruct decoded at 139.6 tok/s and Llama-3.1-8B-Instruct at 130.2 tok/s. Each row in the table is a like-for-like pair: the same hardware, quant and runtime family on the chat-short workload, using each side's fastest such run. Runs can still differ in host, runtime build and context. These are reference results, not a controlled speed or answer-quality ranking. Inspect both source runs before choosing hardware. 1 more hardware config was measured on both but not like-for-like; they are listed separately in the "Not a like-for-like pair" table below and left out of the comparison.
Hardware with data on both models
| Hardware | Qwen2.5-7B-Instruct decode | Llama-3.1-8B-Instruct decode | Δ | Source runs |
|---|---|---|---|---|
| M3 Ultra (60-core GPU) | 139.6tok/s chat-short · mlx · 4bit | 130.2tok/s chat-short · mlx · 4BIT (from file name) | +9.4 | r_5r6rhiynenc · r_v2pbc0rq2l4 |
| M3 Pro (18-core GPU) | 30.52tok/s chat-short · mlx · 4bit | 29.20tok/s chat-short · mlx · 4BIT (from file name) | +1.3 | r_llzv_g-ymaf · r_h0-use1ypnb |
Not a like-for-like pair
These have data on both sides, but the model size, quant or runtime differs, so they are not a head-to-head and are left out of the comparison above.
| Hardware | Qwen2.5-7B-Instruct decode | Llama-3.1-8B-Instruct decode | Why it isn't matched | Runs |
|---|---|---|---|---|
| RTX 5090 (32GB) | 246.7tok/s llama.cpp · quant unreported | 232.2tok/s llama.cpp · quant unreported | quant unreported | r_3yn-4321hp- · r_kfrkg-vn376 |
See also: Qwen2.5-7B-Instruct benchmarks · Llama-3.1-8B-Instruct benchmarks · All hardware · All models · Methodology