RTX 3090 vs 4090 for local LLMs: measured tok/s, 24GB limits

Across 9 matched pairs (same model, size and quant, same runtime family, chat-short), the RTX 4090 decoded 0.8% to 22.5% faster than the RTX 3090, with a median of 14.9%: the smallest lead was Llama 3.1 8B on Ollama (137.32 vs 136.20 tok/s, r_2np7abzjobe vs r_9hurqggbshk) and the largest was GLM-4.7-Flash on Ollama (164.94 vs 134.62 tok/s, r_ted0u8uyre0 vs r_kalwcyab-86). Leads are calculated from single measured runs on different hosts, and the llama.cpp pairs used different llama.cpp builds; see the full measured table for every pair.

A standard RTX 3090 and RTX 4090 both have 24GB of VRAM. Moving from one to the other does not itself give a larger memory budget. If your local model already fits, compare first-token delay and generation speed on the same workload before deciding whether an upgrade is worth its total cost.

On the same model (Qwen3.8-27B Q4_K_M), the same runtime (llama.cpp) and the same chat-short workload, a standard 24GB RTX 4090 measured 43.06 tok/s, against 37.65 tok/s on an RTX 3090. That makes the 4090 about 14% faster (calculated; runs r_11kn1epxewh and r_ptct8o01o5l). Each figure is a single run, on different host CPUs.

For every measured RTX 4090 and RTX 3090 result side by side, see the full RTX 4090 vs RTX 3090 measured table.

Our July 2026 Ollama comparison used 4090 hosts that reported 48GB of GPU memory, which suggests modified cards. We've removed those rows. The comparison below uses only standard 24GB cards. The July 3090 rows stay as 3090-only observations.

NVIDIA lists 24GB GDDR6X for the RTX 3090 and 24GB GDDR6X for the RTX 4090.

Matched test: Qwen3.8-27B on a standard 24GB 3090 and 4090

Both runs were measured on 2026-10-14, using Q4_K_M, llama.cpp 0.6.0-dev and suite-v1. Each is a single run. All values are measured.

Hosts: 3090 on an EPYC 7C13 with 1008GB (llama.cpp commit 8c2134a); 4090 on an EPYC 7642 with 252GB (commit df18bee).

With a long prompt, the 4090 reached its first token in 1,684 ms vs 2,921 ms on the 3090, so prompt processing widens the gap more than generation speed does (measured, same runs).

For reference, Qwen2.5-Coder-14B on a 24GB RTX 4090 measured 89.87 tok/s chat-short with llama.cpp (r_eqnsqmgz29a). It's not comparable to the 3090's 69.21 on Ollama, because the runtimes differ.

Matched Ollama test: the same three models on a standard 24GB 4090

To replace the removed 48GB rows, we ran the same Ollama 0.31.1 tags (Q4_K_M) on a standard 24GB RTX 4090 on October 14, 2026, and paired each with the July RTX 3090 run of the same model and workload. All values are measured decode tok/s and first-token time; each is a single run, on different hosts.

* That 3090 request includes a cold model load (see below), so its first-token time and, for Qwen 7.6B, its short-chat decode are not a fair GPU comparison. Hosts: 3090 on EPYC 7663 (7.6B/8B) or EPYC 7702P (14.8B), 252GB; 4090 on EPYC 7642, 252GB.

On the 8B-class models the two cards decoded at nearly the same speed (137.32 vs 136.20 tok/s on Llama 3.1 chat-short; 133.15 vs 135.03 on Qwen 7.6B chat-long). On the 14.8B model the 4090 was about 17% faster on chat-short (81.18 vs 69.21) and about 16% faster on chat-long (76.65 vs 65.83), calculated from the measured runs. The gap growing with model size is consistent with the 4090's higher memory bandwidth mattering more on larger models; that is our interpretation, not a separate measurement.

RTX 3090 short-chat observations (July 2026, Ollama)

These three RTX 3090 runs were received July 2, 2026, using suite-v1, client 0.0.3 and Ollama 0.31.1. All selected rows report Q4_K_M and batch size one. Model sizes are the reported 7.6B, 8.0B and 14.8B values. The table retains both fast and slow requests rather than choosing a model's highest rate from different workloads.

Scroll each table sideways to see timings and token counts.

The 3090 Qwen 7.6B short request reports 39.86 tok/s, while its concurrent-decode row reports 139.16. The older version of this guide used the latter alongside short-chat results for other models. Those are different workloads; the headline 15–30% faster was not a dependable general GPU comparison.

Why did a 136 tok/s result still wait almost 50 seconds?

The 3090 Llama short request generated at 136.20 tok/s after a 49.960-second first-token wait. Its backend fields report about 26.17 seconds loading and 23.78 seconds evaluating the prompt. That wait is mostly model loading, not generation speed.

The 3090 Qwen 7.6B short request likewise reports roughly 22.86 seconds loading and 23.47 seconds prompt evaluation. We have not identified the cause of every timing difference. Do not treat those first requests as a universal 3090 penalty, discard them silently, or average loading delays into an unrelated warm-response comparison.

For your setup, record the first request separately, warm the model, then repeat the same input and output cap several times while keeping cache policy explicit. Our loading and latency checklist explains how to distinguish loading, prompt processing, queuing and generation.

What happened with a longer prompt?

These inputs contain 3,137 or 3,168 tokens. Each row is still a single observation. The reported rates describe these requests; they do not certify performance at 32K or 128K context, concurrent serving, coding quality or equal end-to-end task latency.

The 3090 7.6B/8B records identify an EPYC 7663 host with 252GB system RAM; the 14.8B record identifies an EPYC 7702P host with 252GB. Exact model digests, common power limits, controlled background load, verified GPU placement and repeated trials are not established by these selected records.

Download the reviewed source snapshot. It retains all four workload rows for each cited run, including the original timing fields. The tables select chat-short and chat-long; the receipt date is not proof of the exact execution date.

Will the 4090 fit a larger model than the 3090?

With the same 24GB capacity, exact model artifact, cache precision and runtime, changing the GPU name does not create another 24GB. Budget for weights, KV or recurrent state, runtime buffers and other GPU users together. Parameter count or download size alone cannot establish whether your intended context fits fully in VRAM.

Use the Qwen3.8 24GB planning example to inspect exact-file inputs and context assumptions. It is an estimate, not a measured 3090/4090 fit result. The Qwen model page explains the hybrid-cache calculation and its limits.

When is paying more for the 4090 justified?

• You already own a 3090 and the workload fits: identify the bottleneck before replacing it. Require a repeated test of the same artifact, runtime, input length, output cap and cache policy on the candidate system.
• You are buying your first local rig: compare current complete-system quotes, used-card condition, warranty, power supply, cooling and case compatibility. This page does not assume a fixed purchase-price ratio or current rental rate.
• Your workload exceeds 24GB: compare a smaller acceptable artifact, lower necessary context, supported offload, or a larger-memory configuration. A stock 3090-to-4090 replacement alone does not solve the capacity gap.
• You are renting: compare total cost per completed task, including setup, model loading, storage and idle time. A quoted hourly GPU price and a rate from an unmatched workload cannot establish the cheaper service.

Only pay for speed that changes your actual workflow. For interactive use, first-token latency and useful answer completion matter alongside tok/s; for queued jobs, check sustained throughput under your intended concurrency. Keep task quality constant when comparing costs.

How to make a more comparable test

Record the exact model tag and digest, quantization, runtime version, actual GPU memory, offload logs, power settings and background load. Separate cold loading from warm requests. Keep prompt text, output cap, thinking mode, context allocation and cache policy fixed, run several repetitions and publish all measurements with failures. A fresh install of the current CLI is not an exact reproduction of these July client-0.0.3 records.

Use the benchmark contribution instructions and inspect a saved local result before submitting it. For newer repeated evidence on a different GPU, see our Qwen3.8 and Gemma 4 RTX 5090 study. The matched 3090/4090 test is the Qwen3.8-27B table above.

Source review: September 16, 2026. This revision corrects mixed-workload, unverified price and nonstandard-host claims in the original July guide. Updated October 14, 2026: we removed the 48GB-labelled 4090 rows and added matched Qwen3.8-27B (llama.cpp) and Ollama tests on standard 24GB cards.