Hardware FAQ
RTX 3090 — frequently asked questions
Direct answers to the questions local-LLM enthusiasts ask about RTX 3090, drawn from 14 signed runs on llm-speed. Every numerical claim links to a verifiable run permalink at /r/<id>.
What's the fastest LLM on RTX 3090?
The fastest measured LLM on RTX 3090 on llm-speed is gpt-oss-20b at 207.2 decode tok/s on llama.cpp 0.7.0-dev (build 1, commit ea0c4c2) (workload chat-short, run r_nakw1nq6clw). Cite as https://llm-speed.com/r/r_nakw1nq6clw.
This is the headline decode tokens-per-second across every (model, backend) pairing submitted on RTX 3090; faster results may exist on hardware not yet benchmarked, but among signed runs this is the published top.
Can I run a 7B-class model on RTX 3090?
Yes — llm-speed has measured Nemotron-3.5-Bolt-74B-A9B-24GB at 87.8 decode tok/s on llama.cpp 0.7.0-dev (build 11669, commit 620d2225c) (workload concurrent-decode, run r_-sifd5otbpy) on RTX 3090, confirming the 7-9B parameter range fits and runs end-to-end on this hardware. Cite as https://llm-speed.com/r/r_-sifd5otbpy.
7B-class models on RTX 3090 typically use the published quant scheme on the run page; check the run for the exact backend version and model digest.
Can I run a 30B-class model on RTX 3090?
Yes — llm-speed has measured Qwen3-Coder-30B-A3B-Instruct at 199.7 decode tok/s on llama.cpp 0.7.0-dev (build 1, commit ea0c4c2) (workload chat-short, run r_0dh1kzbv4h0) on RTX 3090, confirming the 27-33B parameter range, including MoE 30B-A3B variants fits and runs end-to-end on this hardware. Cite as https://llm-speed.com/r/r_0dh1kzbv4h0.
30B-class models on RTX 3090 typically use Q4_K_M; check the run for the exact backend version and model digest.
Can I run a 70B-class model on RTX 3090?
Not at native precision: RTX 3090 has roughly 24 GB of VRAM/unified memory and a 70B-class model (70-72B parameter range) needs at least 48 GB even at 4-bit quantization, so a 70B-class model won't fit end-to-end on RTX 3090 as a single-GPU/SoC config.
For workloads that do fit, see the RTX 3090 leaderboard for the largest model successfully measured on this hardware.
How does RTX 3090 compare to RTX 4090 for local LLM inference?
For a head-to-head between RTX 3090 and RTX 4090, see the side-by-side comparison page at https://llm-speed.com/vs/rtx-3090-vs-rtx-4090, which lays out every (model, backend) pair where both rigs have a signed run.
As a single-rig anchor, RTX 3090 tops out at 207.2 decode tok/s on gpt-oss-20b via llama.cpp (run r_nakw1nq6clw); the RTX 4090 top number is on its own /hw/<slug> page so the comparison stays grounded in measured numbers, not extrapolation.
RTX 3090 vs RTX 4090 (side-by-side) · RTX 4090 leaderboard · RTX 3090 leaderboard
Which backend is fastest on RTX 3090?
Of the backends with a signed local run on RTX 3090, llama.cpp leads at 207.2 decode tok/s on gpt-oss-20b (run r_nakw1nq6clw), followed by ollama at 189.5 decode tok/s on coder-v2 (run r_o2-1w665rtq).
These rows are not a same-model head-to-head — they're each backend's best result across any model on RTX 3090. For a true llama.cpp-vs-ollama comparison, run "llm-speed bench --backends llama.cpp,ollama --models <model>" on a RTX 3090 machine.
llama.cpp: run r_nakw1nq6clw · ollama: run r_o2-1w665rtq · RTX 3090 leaderboard
How much VRAM do I need for local LLM inference on RTX 3090?
RTX 3090 has configurations with up to 24 GB of VRAM, which sets the ceiling on what fits: a 7B-class model at 4-bit needs roughly 6 GB, a 30B-class model at 4-bit needs roughly 18 GB, and a 70B-class model at 4-bit needs roughly 38 GB.
Long-context workloads add KV-cache pressure on top of the weights — a 32k-token context on a 70B-class model adds another 8-16 GB depending on attention layout, so the practical fit is tighter than the weight-only number.
For exact "did it actually run" evidence on RTX 3090, the leaderboard at /hw/rtx-3090 lists every model that has produced a signed result on this hardware.
Is RTX 3090 worth it for local LLM inference in 2026?
On signed data, RTX 3090 delivers up to 207.2 decode tok/s on gpt-oss-20b via llama.cpp (run r_nakw1nq6clw), which puts it comfortably above interactive-coding thresholds (~80 tok/s) for the published top configuration.
"Worth it" depends on your model class: RTX 3090 is most useful for 7B-30B class local models with comfortable headroom.
For "what fits and how fast", the per-model rows on /hw/rtx-3090 are the honest answer; for cross-rig comparisons, see /vs/rtx-3090-vs-rtx-4090.
What quantization should I use on RTX 3090?
On RTX 3090, the top decode rates by quant are Q4_K_M at 207.2 tok/s (gpt-oss-20b, run r_nakw1nq6clw) and Q4_0 at 189.5 tok/s (coder-v2, run r_o2-1w665rtq); pick the smallest quant whose quality your task tolerates, since on memory-bandwidth-bound hardware decode speed scales roughly inversely with bytes-per-weight.
Quant choice is a quality-vs-speed tradeoff that this hardware FAQ does not arbitrate; llm-speed publishes hardware-side speed, not output quality. For quality scores, see the model card on Hugging Face and the LMSYS Chatbot Arena.
Q4_K_M: 207.2 tok/s (run r_nakw1nq6clw) · Q4_0: 189.5 tok/s (run r_o2-1w665rtq)