Apple Silicon vs NVIDIA for Local LLMs: Measured Speed vs Memory
Measured: a 96GB M3 Ultra ran Llama-3.3-70B at 16.8 tok/s (MLX); an RTX 5090 ran gpt-oss-20b at 318.4 tok/s (llama.cpp). Different runtimes, not matched.
A high-memory Mac's measured strength is capacity; an NVIDIA card's is decode speed on models that fit in its VRAM. On capacity, the 96GB M3 Ultra ran Llama-3.3-70B 4-bit at 16.8 tok/s with MLX (r_sx3a4y9n-m4), a model that doesn't fit in the RTX 5090's 32GB of VRAM. That capacity point holds whatever the runtime. The RTX 5090 decoded gpt-oss-20b at 318.4 tok/s with llama.cpp (quant not recorded, r_b9ul-vxh9sc); separately, the M3 Ultra decoded gpt-oss-20b at 152.7 tok/s with MLX and an MXFP4 conversion (r_3ijun8ltjnb). Those two speeds aren't directly comparable: the runtime and the 4-bit format differ, so this isn't a matched-runtime comparison. We haven't yet measured llama.cpp on the Mac. The table below lists every model measured on both platforms, each on its own runtime (llama.cpp or Ollama on NVIDIA, MLX on the Mac).
Models measured on both platforms: 17, each on its own runtime (llama.cpp or Ollama on NVIDIA, MLX on Apple). The rows are not directly comparable, and we haven't yet measured llama.cpp on the Mac.
Apple Silicon and NVIDIA's discrete GPUs approach local LLMs from opposite directions. Per manufacturer specs, an M3 Ultra Mac Studio offers up to 512 GB of unified memory at about 819 GB/s, while an RTX 5090 has 32 GB of GDDR7 at about 1.8 TB/s. Single-request decode is largely memory-bandwidth-bound, so a model that fits on the NVIDIA card usually decodes faster there, while the Mac can hold much larger models. The table lists every model we have measured on both platforms; each side used its own runtime, so the rows are not a matched-runtime comparison.
Where the Mac wins: model size
A 70B model at 4-bit needs more memory than a 32GB card has (manufacturer spec), so a single RTX 5090 cannot hold it entirely on the GPU. The M3 Ultra (96GB unified memory) ran these 70B-class and 80B MoE models with MLX 0.31.3 on our chat-short workload. Note the first-token waits of several seconds.
| Reported hardware | Model · quant | Backend · workload | Decode (measured) | TTFT (measured) | Run |
|---|---|---|---|---|---|
| M3 Ultra (60-core GPU) | llama-3.3-70b-Instruct-4bit quant not reported | mlx 0.31.3 chat-short | 16.78tok/s | 5,420ms | r_sx3a4y9n-m4 2026-04-28 |
| M3 Ultra (60-core GPU) | qwen2.5-72b-Instruct-4bit quant not reported | mlx 0.31.3 chat-short | 16.31tok/s | 5,635ms | r_5c80gthqlh6 2026-04-28 |
| M3 Ultra (60-core GPU) | Qwen3-Next-80B-A3B-Instruct-MLX-4bit 4bit | mlx 0.31.3 chat-short · MoE, about 3B active | 80.34tok/s | 4,493ms | r_1pl79r50ofy 2026-05-12 |
Why a Mac takes longer to start answering
First-token wait depends on prompt processing (prefill), which grows with input length. In our Qwen3.8-27B study on an M5 Max, the measured median first-token wait rose from 8.17 s at 840 input tokens to 113.21 s at 12,263 tokens. We have not yet run the matching long-context sweep on an RTX 5090. See decode vs prefill.
How to decide
- Your models fit in 24 to 32GB and you want the fastest replies: an NVIDIA card. Its llama.cpp runs above were faster than the Mac's MLX runs, but that is not a matched-runtime test.
- You want 70B-class or larger models on one quiet machine: a high-memory Mac, accepting slower decode and longer first-token waits.
- Long prompts (big documents, coding agents): budget for prefill time on the Mac, and test your real context length first.
What we have not measured yet
A controlled same-day pair using the same GGUF quant via llama.cpp on both platforms; RTX 4090 vs M4 Max or M5 Max on the same models; and a long-context prefill sweep on the RTX 5090. Run the suite to add a row.
FAQ
Is an RTX 5090 faster than an M3 Ultra for LLMs?
We can't say from a matched test yet. The RTX 5090 decoded gpt-oss-20b at 318.4 tok/s with llama.cpp, and separately the M3 Ultra decoded it at 152.7 tok/s with MLX (MXFP4 conversion). The runtime and 4-bit format differ, so the two aren't directly comparable; we haven't yet measured llama.cpp on the Mac. What is runtime-independent: the M3 Ultra holds 70B models that a 32GB card can't.
Can a Mac run 70B models?
Yes. Measured on an M3 Ultra with 96GB: Llama 3.3 70B 4-bit at 16.8 tok/s and Qwen2.5 72B 4-bit at 16.3 tok/s (MLX, chat-short).
Is unified memory the same as VRAM?
Not exactly. Unified memory is shared by the CPU and GPU, so a Mac can give a large share of it to a model, while VRAM is dedicated to the GPU. Capacities and bandwidths quoted on this page are manufacturer specs, not measurements.
Related: RTX 5090 vs M3 Ultra · side-by-side · best Mac for LLMs · best GPU for local LLMs
Submitted benchmarks
Side-by-side comparisons
See also: All hardware · All models · Methodology