r1
16 workload results across 1 hardware configuration.
Fastest local config
133.8 decode tok/s
on RTX 4090 (24GB) + AMD EPYC 75F3 32-Core Processor (64c) + 504GB via ollama (Q4_K_M). see full run
Local runs (16 workload results)
Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.
RTX 4090 (24GB) + AMD EPYC 75F3 32-Core Processor (64c) + 504GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_zrjj-pj93q8 |
| chat-long | ollama@0.31.1 | Q4_K_M | 81.09tok/s | 240.4tok/s | 13,070ms | r_zrjj-pj93q8 |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_zrjj-pj93q8 |
| agent-trace | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_zrjj-pj93q8 |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_fg77v2hhohb |
| chat-long | ollama@0.31.1 | Q4_K_M | 131.9tok/s | 686.3tok/s | 4,578ms | r_fg77v2hhohb |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_fg77v2hhohb |
| agent-trace | ollama@0.31.1 | Q4_K_M | 133.8tok/s | 8,852.4tok/s | 366ms | r_fg77v2hhohb |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_l2ck67ls40i |
| chat-long | ollama@0.31.1 | Q4_K_M | 3.76tok/s | 14.28tok/s | 219,986ms | r_l2ck67ls40i |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_l2ck67ls40i |
| agent-trace | ollama@0.31.1 | Q4_K_M | 3.79tok/s | 3,515.1tok/s | 971ms | r_l2ck67ls40i |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_2r1p3w9ps7c |
| chat-long | ollama@0.31.1 | Q4_K_M | 131.0tok/s | 599.9tok/s | 5,238ms | r_2r1p3w9ps7c |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_2r1p3w9ps7c |
| agent-trace | ollama@0.31.1 | Q4_K_M | 132.4tok/s | 9,612.8tok/s | 337ms | r_2r1p3w9ps7c |
Community folklore
43 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.
- communityconfidence 70%
2.13tok/s — DeepSeek R1 on RTX 5090 via llama.cpp
our signed data: RTX 5090 · DeepSeek R1
“ok/sec *without* GPU on local gaming rig! Don't rush out and buy that 5090TI just yet (if you can even find one lol)! I just inferenced \~2.13 tok/sec with 2k context using a dynamic quant of the full R1 671B model (not a distill) after *disabling* my 3090TI GPU on a 96GB RAM g…”
- communityconfidence 70%
2.00tok/s — DeepSeek R1 on RTX 5090 via llama.cpp
our signed data: RTX 5090 · DeepSeek R1
“DeepSeek R1 671B over 2 tok/sec *without* GPU on local gaming rig! Don't rush out and buy that 5090TI just yet (if you can even find one lol)! I just inferenced \~2.13 t”
- communityconfidence 60%
18.43tok/s — DeepSeek R1 on M3 Ultra via mlx
our signed data: M3 Ultra · DeepSeek R1
“R1 671B Q4 - M3 Ultra 512GB with MLX🔥 Yes it works! First test, and I'm blown away! Prompt: "Create an amazing animation using p5js" * **18.43 tokens/sec** * Generates a p5js zero-shot, tested at video's end * Video in real-time, no acceleration! https://reddit.com/link/1j9vj…”
- communityconfidence 60%
7.00tok/s — deepseek-r1 on M3 Max via ollama
our signed data: M3 Max · deepseek-r1
“oubles the electricity cost compared to RTX 6000 ADA (19tps) or RTX A6000 (12tps). \- \`M3 Max 40GPU\` has high memory but only delivers 3-7 tps for \`deepseek-r1:70b\`. It is also loud, and the GPU temperature is high (> 90 C). https://preview.redd.it/8r7cwajfn9ee1.png?width=1…”
- communityconfidence 60%
12.00tok/s — deepseek-r1 on RTX A6000 via ollama
our signed data: RTX A6000 · deepseek-r1
“2 x RTX 3090 (17tps) in terms of price/performance. However, it doubles the electricity cost compared to RTX 6000 ADA (19tps) or RTX A6000 (12tps). \- \`M3 Max 40GPU\` has high memory but only delivers 3-7 tps for \`deepseek-r1:70b\`. It is also loud, and the GPU temperature is …”
- communityconfidence 60%
19.00tok/s — deepseek-r1 on RTX 6000 Ada via ollama
our signed data: RTX 6000 Ada · deepseek-r1
“` performs best with 2 x RTX 3090 (17tps) in terms of price/performance. However, it doubles the electricity cost compared to RTX 6000 ADA (19tps) or RTX A6000 (12tps). \- \`M3 Max 40GPU\` has high memory but only delivers 3-7 tps for \`deepseek-r1:70b\`. It is also loud, and th…”
- communityconfidence 60%
17.00tok/s — deepseek-r1 on RTX 3090 via ollama
our signed data: RTX 3090 · deepseek-r1
“\~24GB VRAM: RTX 3090 offers the best price/performance. RTX Titan is acceptable. \- \`deepseek-r1:70b\` performs best with 2 x RTX 3090 (17tps) in terms of price/performance. However, it doubles the electricity cost compared to RTX 6000 ADA (19tps) or RTX A6000 (12tps). \- \`M…”
- communityconfidence 60%
14.10tok/s — DeepSeek-R1 on RTX 3090 via llama.cpp
our signed data: RTX 3090 · DeepSeek-R1
“I can't run it fully in Q8 yet, that’s a good excuse to buy another 3090! For now, let's use Q6. With some tuning, we can go **from 12.4 to 14.1 tokens per second**. Not bad. Then we move on to a 70B model. I'm using **DeepSeek-R1-Distill-Llama-70B** in Q4. We start at **10.3** …”
- communityconfidence 60%
4.10tok/s — deepseek-R1 on L40S via ollama
our signed data: L40S · deepseek-R1
“ompt eval using GPT-OSS 120B in ollama, with no special hackery. For comparison, the much smaller deepseek-R1 70B takes the same prompt at 4.1 t/s and 1173 t/s eval and prompt eval respectively on this system. Even on an L40 which can load it totally into VRAM, R1-70B only hits…”
- communityconfidence 60%
10.57tok/s — DeepSeek r1 on M3 Max via mlx
our signed data: M3 Max · DeepSeek r1
“fastest mammal and got: R1-Distill-Qwen-32B-MLX-4bit - 19.00 tok/sec - 654 tokens - 0.67s to first token R1-Distill-Qwen-32B-MLX-8bit - 10.57 tok/sec -82 tokens - 0.30s to first token R1-Distill-Qwen-32B-Q4\_K\_M-GGUF - 15.93 tok/sec - 744 tokens - 0.73s to first token R1-Di…”