RTX 3090 in 2026: measured local LLM speed

On Llama 3.1 8B (Ollama 0.31.1, Q4_K_M, chat-short), the RTX 3090 generated 136.2 tok/s, within 1% of the RTX 4090's 137.3. On the larger Qwen2.5-Coder 14.8B, the 4090 was about 17% faster (81.2 vs 69.2). (Measured, single runs on different hosts; both cards have 24 GB.)

The RTX 3090 is the most-argued-about GPU in local-LLM communities: a 24 GB card with endless claims about how fast it runs models. Almost none of those numbers are verifiable. So we rented one and measured it under our signed, reproducible suite; every figure below links to the exact run.

What a 3090 actually does

On an RTX 3090 (24 GB), Q4 via Ollama: DeepSeek-Coder-V2-16B (a mixture-of-experts) decodes 189 tok/s, Llama-3.1-8B 136 tok/s, and the dense Qwen2.5-Coder-14B 69 tok/s, all on the chat-short workload. Qwen2.5-Coder-7B reached 139 tok/s on the concurrent-decode workload (its chat-short row includes a 46 s model load, so we don't quote it). Model sizes are the Ollama tags' reported sizes in each run (7.6B, 8.0B, 14.8B, 15.7B). Full, current standings are on /hw/rtx-3090 and the cheatsheet.

The MoE surprise

The fastest coder on the card is also one of the largest. DeepSeek-Coder-V2 is a 16B mixture-of-experts with only ~2.4B parameters active per token, so it decodes 189 tok/s, nearly 3x the dense Qwen2.5-Coder-14B at 69 tok/s on the very same 3090. On a memory-bandwidth-bound card, a small-active-parameter MoE is the sweet spot: big-model quality at small-model speed.

Is that fast?

About 137 tok/s is roughly 5x quicker than you can read, with zero per-token cost and everything staying on your machine. The 24 GB of VRAM comfortably holds a 7-14B coder at Q4 with room for context. The tok/s thresholds are on the glossary.

The value case

The 3090 has the same 24 GB as an RTX 4090. On the same model and runtime, it measured Qwen3.8-27B Q4_K_M at 37.65 tok/s vs 43.06 on a 24GB 4090 (llama.cpp, chat-short), about 14% slower (calculated). It is not the quickest card we measure (that is the RTX 5090). Whether it is the value pick depends on the price you can get today, so check current listings.

Reproduce it, or add your rig

Every number here is one command from reproduction:

$ pipx install llm-speed && llm-speed bench

Numbers as of July 2026; the linked runs and the cheatsheet always reflect current data. Method: /methodology.