Best GPU for local LLMs under $2,000: measured RTX 3090 vs RTX 4090

Same model on both cards (measured)

4090 vs 3090 (calculated): about 14% faster decode on Qwen3.8-27B. Across all four workloads of the same runs, the RTX 3090 measured 37.65 / 37.25 / 37.53 / 36.75 tok/s and the RTX 4090 43.06 / 42.63 / 43.00 / 42.07 (chat-short, chat-long, concurrent-decode, agent-trace), so the gap holds at about 14%.

Prefill and first token: chat-short prefill was 368.5 tok/s on the 3090 vs 428.6 on the 4090, and chat-long time to first token was 2,921 ms vs 1,684 ms (measured). The 4090's lead is larger when prompts are long.

Older Ollama runs from a host whose RTX 4090 reports 48GB of GPU memory are left out of this comparison: a standard RTX 4090 has 24GB, and we can't confirm what that card is.

What else runs on a 24GB RTX 4090 (measured)

Qwen3-32B Q4_K_M ran at 43.47 tok/s, Qwen2.5-Coder-14B at 89.87, Qwen3-Coder-30B-A3B at 232.5 and gpt-oss-20b at 242.6. MoE models with few active parameters decoded 5× or more faster than the dense 27–32B models on the same card.

The free speedup is software, not a pricier card

With MTP (multi-token prediction, a form of speculative decoding) on, the same RTX 4090 decoded Qwen3.8-27B at 84.51 tok/s vs 43.06 with it off (r_7j8rpk9mwhg vs r_11kn1epxewh; calculated ≈1.96×). That is faster than our RTX 5090 without MTP (65.73, r_c3ps9wjygi9). The two use different builds and hosts, so it's a reference point, not a controlled test. MTP also costs prefill speed and first-token time; see our measured MTP post.

How to choose

• NVIDIA launch prices (launch MSRP, not today's price; check current listings yourself): RTX 3090 $1,499 (NVIDIA newsroom, 2020-09-01); RTX 4090 $1,599 (nvidia.com, available 2022-10-12).
• Pick the 3090 if 24 GB is what you need and you can buy one well below a 4090. Measured, it gives up about 14% decode on a 27B model.
• Pick the 4090 for faster prefill on long prompts and about 14% more decode on Qwen3.8-27B.
• Neither fits a 70B at Q4 in 24 GB. For scale, Qwen3.8-27B used 18.62 GiB on our 5090 run. Check your model and context with the memory-fit estimator.

Not measured, or not in this tier

• RTX 5090: excluded. NVIDIA's launch price was $1,999 (NVIDIA newsroom, Jan 2025), but we have no dated, sourced current price under $2,000. See RTX 5090 vs RTX 4090.
• RTX 5080 16GB, RX 7900 XTX, Arc B580, Radeon AI PRO R9700, dual RTX 3090: not measured.

FAQ

What is the cheapest GPU for running local LLMs well?

Of the cards we've measured, the RTX 3090 (24GB) is the lower-cost option. It decoded Qwen3.8-27B Q4_K_M at 37.65 tok/s, vs 43.06 on an RTX 4090 (measured, llama.cpp). We haven't measured cheaper 12–16GB cards yet.

Is the RTX 4090 worth it over a 3090 for LLMs?

On Qwen3.8-27B the 4090 was about 14% faster at decode and faster at prefill, with the same 24 GB. Whether that's worth it depends on the price gap you see.

Can a 24GB card run a 27B or 32B model?

Yes, at 4-bit. Our 4090 ran Qwen3.8-27B at 43.06 tok/s and Qwen3-32B at 43.47 tok/s, both Q4_K_M with every layer on the GPU (measured, llama.cpp -ngl 99).

Can a 24GB card run a 70B model?

Not fully on the GPU at Q4. Use the memory-fit estimator for your model and context.

How do I make a 3090 or 4090 faster without buying a new card?

Turn on MTP or speculative decoding if your model supports it. On our 4090, Qwen3.8-27B went from 43.06 to 84.51 tok/s (measured, chat-short), at the cost of slower prefill and a later first token.

Why isn't the RTX 5090 here?

We couldn't source a dated current price under $2,000. Its results are on RTX 5090 vs RTX 4090.

Related: RTX 3090 · RTX 4090 · RTX 4090 vs RTX 3090 · Qwen3.8-27B · RTX 3090 value pick · best GPU 2026 · cheatsheet · methodology