Best GPU for Local LLMs (2026): Measured tok/s on RTX 5090, 4090, 3090

Three checks before buying or adding a GPU

Before spending, run representative prompts at your intended context and concurrency. Record first-token wait, decode speed, memory use and whether the answers meet your task. Our Qwen3.8 and Gemma 4 RTX 5090 study supplies specific speed measurements; it does not validate a different GPU pair, long-context concurrency or coding quality.

Sources checked September 22, 2026. The examples below retain reported host capacities, including nonstandard configurations. They do not prove GPU-only residency, current prices or which purchase has the best value. Token counts describe measured prompts and responses, not the maximum configured context. Missing fields remain explicit.

Short answer

On our measured runs, the RTX 5090 (32GB) is the fastest consumer card we have tested: 318.4 tok/s on gpt-oss-20b (run) and a median of 65.66 tok/s on dense Qwen3.8-27B Q4_K_M over three repeats (study), all measured with llama.cpp on the chat-short workload. A 24GB RTX 4090 or RTX 3090 measured 43.06 and 37.65 tok/s on the same Qwen3.8-27B Q4_K_M (llama.cpp, chat-short). Pick by the largest model you need to keep fully on the GPU, then by speed.

Not a controlled comparison: hosts, llama.cpp builds and dates differ, and some rows below use Ollama 0.31.1. Runs from a host whose RTX 4090 reports 48GB of GPU memory (a retail 4090 has 24GB) are left out. All rows are single-request decode speeds. See methodology.

Measured speeds by model size

8B to 9B dense

14B to 20B (dense and MoE)

27B to 32B dense

30B-class MoE

24GB vs 32GB: what the extra memory buys

In our RTX 5090 study, Qwen3.8-27B Q4_K_M (an 18.97 GB file) ran with all 65 layers on the GPU and the runtime reported 18.62 GiB allocated for model, context and compute buffers, with a 16,384-token context configured. On 24GB cards we have since measured the same model with every layer on the GPU (llama.cpp -ngl 99): 43.06 tok/s on an RTX 4090 (run) and 37.65 on an RTX 3090 (run), llama.cpp chat-short. We have also measured on a 24GB RTX 4090 is Gemma 3 27B Q4_K_M at 47.0 tok/s (run) and the 30B MoE qwen3-coder at 145.2 tok/s (run), both via Ollama. Card memory sizes are manufacturer specs. Use the memory-fit estimator for your exact file and context.

Is a used RTX 3090 still worth it?

For models up to about 16B, our measured 3090 rows are fast: 189.5 tok/s on deepseek-coder-v2 (15.7B, Q4_0) and 136.2 tok/s on llama3.1 8B Q4_K_M. The 14.8B qwen2.5-coder ran at 69.2 tok/s. These come from one host on Ollama 0.31.1, so treat them as examples rather than a ranking. Same-backend runs of the same models on a 3090, 4090 and 5090 are the comparison we still need.

What we have not measured yet

A controlled same-host, same-build comparison of the RTX 3090, 4090 and 5090; and any RTX 5080, RTX 5070 Ti or RX 7900 XTX run. If you own one, run the suite and submit: your row will appear here after the next build.

FAQ

What is the fastest GPU for local LLMs?

Of the GPUs we have measured, the RTX 5090: 318.4 tok/s on gpt-oss-20b and 259.9 tok/s on Qwen3-Coder-30B-A3B (llama.cpp, chat-short).

How many tok/s does an RTX 5090 get on a 27B model?

Measured: a median of 65.66 tok/s for Qwen3.8-27B Q4_K_M (three repeats, llama.cpp 9725a31, chat-short), and 74.0 tok/s for Qwen3.6-27B Q4_K_M in a single earlier run.

Is 24GB of VRAM enough for a 27B model?

Yes at 4-bit, in our runs: Qwen3.8-27B Q4_K_M ran with every layer on the GPU (-ngl 99) at 43.06 tok/s on a 24GB RTX 4090 and 37.65 tok/s on a 24GB RTX 3090 (measured, llama.cpp, chat-short).

Related: RTX 5090 vs RTX 4090 · Qwen3.8-27B · Apple Silicon vs NVIDIA · cheatsheet