Qwen3.8-27B speed and VRAM: measured on RTX 5090
18 workload results across 4 hardware configurations.
On Qwen3.8-27B Q4_K_M (llama.cpp, chat-short), we measured the RTX 5090 at 65.66 tok/s, the RTX 4090 at 43.06 and the RTX 3090 at 37.65 (runs r_dz5a3lzxhrj, r_11kn1epxewh and r_ptct8o01o5l). The cards ran on different hosts and llama.cpp builds, so treat these as measured reference points, not a controlled ranking.
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run |
|---|---|---|---|---|---|
| RTX 5090 (32GB) | Qwen3.8-27B Q4_K_M | llama.cpp 1 (9725a31) chat-short · RTX 5090 32GB (median of three repeats) | 65.66tok/s | 118ms | r_dz5a3lzxhrj 2026-09-05 |
| RTX 4090 (24GB) | Qwen3.8-27B Q4_K_M | llama.cpp 0.6.0-dev (build 1, commit df18bee) chat-short · RTX 4090 24GB | 43.06tok/s | 359ms | r_11kn1epxewh 2026-10-14 |
| RTX 3090 (24GB) | Qwen3.8-27B Q4_K_M | llama.cpp 0.6.0-dev (build 1, commit 8c2134a) chat-short · RTX 3090 24GB | 37.65tok/s | 418ms | r_ptct8o01o5l 2026-10-14 |
Original test · 5 September 2026 · three repetitions
How fast is Qwen3.8-27B on an RTX 5090?
We measured 65.7 tokens per second in short chat with Q4_K_M quantization, using one RTX 5090 with 32 GB VRAM and llama.cpp. Median time to the first token was 117.5 ms with the model already loaded. These are single-user, thinking-off results.
| Workload | Decode tok/s | First token (ms) | Prompt tokens | Output tokens |
|---|---|---|---|---|
| chat-short | 65.7 (65.4–65.7) | 117.5 (117.1–131.7) | 114 | 256 |
| chat-long | 65.1 (65.1–65.1) | 909.4 (908.0–910.5) | 3,184 | 874 |
Does it fit in 32 GB of VRAM?
Yes, this tested configuration offloaded all 65/65 layers to the GPU. The runtime reported 18.62 GiB (19,071 MiB) of GPU allocation for model, context and compute buffers combined. The model file is 18.97 GB on disk.
Allocation is from the runtime log, not a measurement of peak whole-device memory. A 16,384-token context was configured; the prompts above do not test a full 16K context. Other idle services remained GPU-resident.
What do these results tell you?
This is a baseline for local text generation with one active request. It helps estimate response speed at the tested prompt lengths. Prompt caching and thinking were disabled; the model stayed loaded between runs.
It does not establish answer quality, coding accuracy, vision performance, cold-start time or multi-user throughput. Different quantization, context lengths and runtime optimizations need separate measurements.
Estimate memory on another GPU or Mac → Estimates are separate from the measured allocation above.
Tested settings and reproducibility
Hardware: RTX 5090 (32 GB), AMD Ryzen 7 9850X3D, 30 GB system RAM reported by WSL. Runtime: llama.cpp commit 9725a31 with CUDA.
Artifact: Qwen3.8-27B-Q4_K_M.gguf, pinned revision 0669b98607d47046c7c2b3f801011d54a08cfccf. SHA-256: 31629f53165ab6a7dad8c9847dcfd1fdf55829dac1e6e748f4a68581b0033d34.
Server configuration (use a free local port and the path to your verified model file):
llama-server --model Qwen3.8-27B-Q4_K_M.gguf --alias Qwen3.8-27B-Q4_K_M.gguf \
--host 127.0.0.1 --port 8082 --n-gpu-layers 999 \
--ctx-size 16384 --parallel 1 --no-cache-prompt --cache-ram 0 \
--reasoning off --temp 0 --seed 42Measurements used suite-v1 chat-short and chat-long, with output caps of 256 and 1,024 tokens. The table reports actual token counts. Decode speed and first-token latency use the benchmark client's stream timing.
The client was an isolated copy of llm-speed 0.0.5 with fixes for underscore-containing quantization names and streamed token usage. Prompts and timing formulas were unchanged. Installing the published CLI alone may not reproduce that exact client. The configuration above documents the server; it is not a complete benchmark command.
Five earlier submissions affected by a competing service restart were excluded. Only the three accepted repetitions linked below contribute to this summary.
Inspect the measured runs
Also tested on this GPU: Gemma 4 12B IT QAT. Tokenizers and output lengths differ, so speed alone does not rank the models' usefulness.
Memory planning · sources checked 16 September 2026
Qwen3.8-27B VRAM requirements: 24 GB or 32 GB?
For the exact ggml-org Q4_K_M file we tested, the planning estimate is about 21.9 GB at 32,000 tokens with one text sequence and FP16 attention cache. At 128,000 tokens it rises to about 28.2 GB. A 24 GB card has a different context budget from a 32 GB card, even at the same weight quantization. These are calculations, not measured 24 GB or long-context fit tests.
| Context | FP16 KV | Total estimate | 24 GB budget | 32 GB budget |
|---|---|---|---|---|
| 4,000 | 0.26 GB | 20.0 GB | Within estimate | Within estimate |
| 32,000 | 2.10 GB | 21.9 GB | Within estimate | Within estimate |
| 128,000 | 8.39 GB | 28.2 GB | Exceeds budget | Within estimate |
The published hybrid attention configuration gives 64 KiB of FP16 attention cache per token. The total adds the file size and a rough 0.8 GB allowance for recurrent state and runtime buffers. File size is not resident VRAM; the allowance is not an upper bound. Other applications, vision, simultaneous requests and different runtimes can change the result. Within estimate means at least 8% of the selected budget remains under this formula; it does not certify a working setup.
Why do Q4 download sizes differ?
Choose the repository, revision and filename before comparing memory claims. These publisher listings are distinct artifacts; Unsloth's UD-Q4_K_M is not the plain Q4_K_M file in our speed test. Smaller files do not establish equal quality or faster inference.
- ggml-org · Qwen3.8-27B-Q4_K_M.gguf
18.97 GB on disk · the file used in our measured 5090 study.
- unsloth · Qwen3.8-27B-UD-Q4_K_M.gguf
16.46 GB on disk · publisher metadata only; not benchmarked here.
- bartowski · Qwen3.8-27B-Q4_K_M.gguf
17.44 GB on disk · publisher metadata only; not benchmarked here.
Download exact bytes, pinned revisions and publisher-reported hashes (JSON)
Before buying or renting a GPU
Match the exact model file, weight precision, cache precision, context and simultaneous users to your workload. Then check the runtime's allocation and offload logs on that setup. Our measured results above establish speed at their stated prompt lengths; they do not prove that the same speed continues at 128K. A cloud GPU is off-premises; check whether that meets your data-location requirements.
Compare a 24 GB RTX 4090 budgetCompare a 32 GB RTX 5090 budget
How fast is Qwen3.8-27B on a 48 GiB M5 Max?
Our September 16, 2026 text-only MLX study completed on a 40-GPU-core M5 Max with 48 GiB unified memory. With a fresh prompt cache, median first-token wait grew from 8.17 to 113.21 seconds as input grew from 840 to 12,263 tokens, while median stream decode stayed around 13–14 tok/s. Fitting the model did not make long prompts quick.
| Actual input tokens | First token (s) | Stream decode (tok/s) |
|---|---|---|
| 840 | 8.17 (3.79–11.78) | 13.70 (11.48–27.17) |
| 3,156 | 27.66 (16.59–35.35) | 13.62 (11.09–25.92) |
| 12,263 | 113.21 (74.39–133.19) | 13.34 (10.10–18.22) |
Exact conversion: mlx-community/Qwen3.8-27B-4bit, revision 3e6447f, using MLX 0.32.1 and mlx-lm 0.31.3. Synthetic document-summary prompts, batch one, thinking off, model already loaded; all responses reached the 256-token cap. These exploratory measurements are separate from signed suite-v1 submissions. The conversion and workloads differ from the RTX 5090 study above, so the two studies do not establish a hardware speed ratio.
Background load was uncontrolled and timings varied substantially. An earlier smoke attempt stopped after swap grew; the completed study does not establish zero paging or a minimum memory tier. It did not test maximum context, concurrent users, image input or answer quality. Prepared-prefix samples are excluded from this table because their request timer excludes prefix preparation.
Full Mac study, memory observations and reproduction methodRaw prompts, outputs and timing events (JSON)All 12 samples (CSV)
Other Qwen 27B results on the RTX 5090
Besides the three Qwen3.8-27B repeats above, we have one earlier measured run of Qwen3.6-27B Q4_K_M on an RTX 5090 with llama.cpp. It is a single run from a different date, with no llama.cpp version recorded, so it is not directly comparable with the Qwen3.8 median. Gemma 4 12B QAT, measured on the same rig and llama.cpp commit as the Qwen3.8 study, is shown for scale: a smaller model decodes faster, which says nothing about answer quality.
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run |
|---|---|---|---|---|---|
| RTX 5090 (32GB) | Qwen3.6-27B-Q4_K_M.gguf quant not reported | llama.cpp chat-short · single run | 73.96tok/s | 190ms | r_f7ulllqu4vk 2026-07-01 |
| RTX 5090 (32GB) | gemma-4-12b-it-qat Q4_0 | llama.cpp 1 (9725a31) chat-short · median of 3 repeats | 139.5tok/s | 61.8ms | r_c2podhe9qwc 2026-09-05 |
What we have not measured yet
- Long context on the RTX 5090: 8K, 32K, 64K and 128K input tokens (decode and first-token wait).
- Higher quants: Q5_K_M, Q6_K and Q8_0.
- q8_0 vs f16 KV cache, and speculative decoding on vs off.
- The same model on a 24GB RTX 4090 or RTX 3090.
If you have one of these setups, run the suite and submit it.
FAQ
How many tokens per second does Qwen 27B get on an RTX 5090? Measured: a median of 65.66 tok/s for Qwen3.8-27B Q4_K_M (chat-short, llama.cpp 9725a31, three repeats) and 74.0 tok/s for Qwen3.6-27B Q4_K_M in a single run.
Does Qwen3.8-27B fit in 32GB of VRAM? Yes at Q4_K_M with a 16,384-token context configured: all 65 layers on the GPU, 18.62 GiB allocated (runtime log).
Is Gemma 4 12B faster than Qwen 27B on a 5090? Yes in decode speed: 139.50 vs 65.66 tok/s median, measured on the same rig. Speed is not quality; see the full comparison.
How fast is Qwen 27B at 128K context? Not measured yet.
Related: RTX 5090 results · same model on an M5 Max · Qwen3.6-27B on an RTX 4090 · best GPU for local LLMs
Local runs (18 workload results)
Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.
RTX 5090 (32GB) + AMD EPYC 9534 64-Core Processor (128c) + 756GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 75.19tok/s | 563.4tok/s | 273ms | r_qfrb2zmyk0k |
| chat-long | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 74.57tok/s | 2,499.5tok/s | 1,290ms | r_qfrb2zmyk0k |
| concurrent-decode | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 75.40tok/s | no data | no data | r_qfrb2zmyk0k |
| agent-trace | llama.cpp@0.7.0-dev (build 11669, commit 620d2225c) | Q4_K_M | 73.52tok/s | 5,065.3tok/s | 436ms | r_qfrb2zmyk0k |
RTX 3090 (24GB) + AMD EPYC 7C13 64-Core Processor (128c) + 1008GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.6.0-dev (build 1, commit 8c2134a) | Q4_K_M | 37.65tok/s | 368.5tok/s | 418ms | r_ptct8o01o5l |
| chat-long | llama.cpp@0.6.0-dev (build 1, commit 8c2134a) | Q4_K_M | 37.25tok/s | 1,103.5tok/s | 2,921ms | r_ptct8o01o5l |
| concurrent-decode | llama.cpp@0.6.0-dev (build 1, commit 8c2134a) | Q4_K_M | 37.53tok/s | no data | no data | r_ptct8o01o5l |
| agent-trace | llama.cpp@0.6.0-dev (build 1, commit 8c2134a) | Q4_K_M | 36.75tok/s | 2,190.5tok/s | 968ms | r_ptct8o01o5l |
RTX 4090 (24GB) + AMD EPYC 7642 48-Core Processor (48c) + 252GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 43.06tok/s | 428.6tok/s | 359ms | r_11kn1epxewh |
| chat-long | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 42.63tok/s | 1,915.1tok/s | 1,684ms | r_11kn1epxewh |
| concurrent-decode | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 43.00tok/s | no data | no data | r_11kn1epxewh |
| agent-trace | llama.cpp@0.6.0-dev (build 1, commit df18bee) | Q4_K_M | 42.07tok/s | 3,644.0tok/s | 629ms | r_11kn1epxewh |
RTX 5090 (32GB) + AMD Ryzen 7 9850X3D 8-Core Processor (8c) + 30GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | llama.cpp@1 (9725a31) | Q4_K_M | 65.66tok/s | 969.8tok/s | 118ms | r_dz5a3lzxhrj |
| chat-long | llama.cpp@1 (9725a31) | Q4_K_M | 65.10tok/s | 3,496.8tok/s | 911ms | r_dz5a3lzxhrj |
| chat-short | llama.cpp@1 (9725a31) | Q4_K_M | 65.73tok/s | 973.6tok/s | 117ms | r_c3ps9wjygi9 |
| chat-long | llama.cpp@1 (9725a31) | Q4_K_M | 65.09tok/s | 3,506.5tok/s | 908ms | r_c3ps9wjygi9 |
| chat-short | llama.cpp@1 (9725a31) | Q4_K_M | 65.43tok/s | 865.3tok/s | 132ms | r_7mod38qsldj |
| chat-long | llama.cpp@1 (9725a31) | Q4_K_M | 65.10tok/s | 3,501.1tok/s | 909ms | r_7mod38qsldj |
Qwen3.8-27B on hardware
Request a Qwen3.8-27B run
No measured run yet for these pairs. Vote for the one you want; the most-requested runs go first. See all requests.
We never sell or share your email. Privacy