Cheapest 70B LLM Rig? Measured Options Compared
Llama-3.3-70B Q4_K_M, rented: two RTX 3090s ran 47% faster (19.1 vs 13.0 tok/s); the A40 cost less per hour and per token at Oct 2026 rates. Runs linked.
The trade-off from our two rented Llama-3.3-70B Q4_K_M runs (llama.cpp): two 24GB RTX 3090s were about 47% faster, and the 48GB A40 cost less per hour. The pair decoded 19.1 tok/s (r_n93ukk0mkmm) and the A40 13.0 tok/s (r_19f64yux97o). RunPod billed the A40 pod at $0.59/hour on Oct 16, 2026 and the two-3090 pod at $1.00/hour on Oct 17, 2026, the rates our bench ledger recorded when each pod started. At those rates and single-stream speeds, a million generated tokens works out to about $12.60 on the A40 and $14.50 on the two 3090s, so the A40 was about 13% cheaper per token (calculated, not measured). The whole test sessions, including setup, cost $0.30 (A40) and $0.25 (two 3090s, a shorter session). Rental prices move, so check current rates before you rent. Each speed is one run on a different host and llama.cpp build. Separately, a 96GB M3 Ultra Mac Studio ran Llama-3.3-70B at 16.8 tok/s with MLX and an MLX 4-bit conversion (r_sx3a4y9n-m4). It isn't directly comparable to the rented runs: the backend and the 4-bit format differ. Owned rigs aren't measured yet. To check which Llama 70B quant fits a given GPU, see Llama 3.3 70B on a GPU: VRAM fit.
Measured 70B-class dense rigs: 2x RTX 3090 (24GB) 19.14tok/s on Llama-3.3-70B-Instruct (llama.cpp, Q4_K_M); A40 (45GB) 12.99tok/s on Llama-3.3-70B-Instruct (llama.cpp, Q4_K_M); M3 Ultra (60-core GPU) 16.78tok/s on mlx-community/llama-3.3-70b-Instruct-4bit (mlx, 4-bit). Runtimes and quant formats differ, so these aren't directly comparable. We have no price basis, so this is not a cheapest ranking.
A 70B-class dense model at 4-bit (Q4_K_M / Q4_0 / 4-bit MLX) lives in roughly 40 GB of memory once you add KV cache for a useful context window. That rules out 24 GB single-card consumer GPUs unless you're willing to split layers, and rules in: a pair of 3090s or 4090s (48 GB combined VRAM), an Apple Silicon Max or Ultra with 64 GB+ unified memory, an AMD MI300X or H100 80 GB on the high end, or AMD's Ryzen AI MAX+ 395 with 96 GB. The buy-once cost ranges from ~$1,500 (used dual 3090) to several thousand for an Ultra Mac Studio. Below is every submitted run for a 70B-or-larger model, ranked by decode tok/s; runs on different runtimes and quant formats aren't directly comparable. If your candidate isn't there yet, run the suite and submit — that row will appear next refresh.
Submitted benchmarks
| Hardware | 70B-class model | decode tok/s | MSRP | Run |
|---|---|---|---|---|
| 2x RTX 3090 (24GB) | Llama-3.3-70B-Instruct | 19.14tok/s | — | r_n93ukk0mkmm |
| M3 Ultra (60-core GPU) | mlx-community/llama-3.3-70b-Instruct-4bit | 16.78tok/s | — | r_sx3a4y9n-m4 |
| A40 (45GB) | Llama-3.3-70B-Instruct | 12.99tok/s | — | r_19f64yux97o |
Side-by-side comparisons
See also: All hardware · All models · Methodology