Local vs hosted: when does buying a GPU pay off?
At low usage, hosted APIs win on $/Mtok. At high sustained usage, a 4090 or M3 Ultra wins. Here's the break-even math, run against live numbers.
We can measure the local side of this decision. A 24GB RTX 4090 decoded gemma3 4.3B at 195.0 tok/s (r_dlanfbgym0h), and an RTX 5090 decoded gpt-oss-20b at 318.4 tok/s (r_b9ul-vxh9sc) for a single request. We haven't measured power draw or sustained duty cycle, so any break-even figure on this page is a calculation from your inputs, not a measurement.
No data submitted for this task yet.
Run the suite to be the first benchmark for this guide:
$ pipx install llm-speed && llm-speed bench
We don't sell hardware and we don't take affiliate commissions on hosted APIs, so the framing is just arithmetic. A consumer GPU's break-even point against a hosted endpoint depends on three things: your sustained decode tok/s, the hosted price per million output tokens, and your duty cycle. Below is a comparison table with each row anchored to a real submitted benchmark.
Submitted benchmarks
| Hardware | Model | decode tok/s | Run |
|---|---|---|---|
| RTX 5880 (48GB) | llama3.2 | 466.9tok/s | r_b948ehgi8g4 |
| RTX 5090 (32GB) | gpt-oss-20b | 324.2tok/s | r_dwp5fgdapmw |
| RTX 4090 (24GB) | gpt-oss-20b | 242.6tok/s | r_t-a9v_ipdit |
| RTX 3090 (24GB) | gpt-oss-20b | 207.2tok/s | r_nakw1nq6clw |
| M3 Ultra (60-core GPU) | mlx-community/DeepSeek-Coder-V2-Lite-Instruct-4bit | 168.3tok/s | r_l_v1-zq_qaz |
| RTX A4000 (16GB) | LFM2.5-24B-A2B | 142.7tok/s | r_j5tgvia-kjk |
| M4 Max (40-core GPU) | qwen3-coder-bench-32k | 113.3tok/s | r_roktphpc--8 |
| M1 Pro (16-core GPU) | llama3.2 | 103.1tok/s | r_w-p7v--eync |
| M5 Max (40-core GPU) | qwen3.6-agent-q6k-ctx128k | 88.29tok/s | r_udwgba7udqu |
| M3 Pro (18-core GPU) | mlx-community-Qwen2.5-7B-Instruct-4bit | 30.52tok/s | r_llzv_g-ymaf |
| 3x A100 (80GB) | Muse-Spark-Open-816B-A42B-UD-00001-of-00005 | 26.30tok/s | r_fsd9qjmmna0 |
| AMD64 Family 23 Model 104 Stepping 1, AuthenticAMD (6c) | llama3.2 | 23.88tok/s | r_b1rn5gi6b3b |
See also: All hardware · All models · Methodology