RTX 5880 LLM benchmark
On the RTX 5880, measured single-user chat decode speeds are: ~1.2B models at 466.9 tok/s (Q8_0, ollama; Measured, 1 result, source run). Short-chat workload, local runtimes, dense models only; quantization, backend and settings vary between submissions, so check each row in the tables below.
Fastest single result in these submissions (any model size)
466.9 decode tok/s
llama3.2 (1.2B) via ollama (Q8_0). Workload: chat-short. Reported hardware: RTX 5880 (48GB) + AMD EPYC 9T24 96-Core Processor (16c) + 62GB. see full run
llama3.2
| Workload | Model size | Reported hardware | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|---|---|
| chat-short | 1.2B | RTX 5880 (48GB) + AMD EPYC 9T24 96-Core Processor (16c) + 62GB | ollama@0.34.2 | Q8_0 | 466.9tok/s | 44.32tok/s | 2,821ms | r_b948ehgi8g4 |
| chat-long | 1.2B | RTX 5880 (48GB) + AMD EPYC 9T24 96-Core Processor (16c) + 62GB | ollama@0.34.2 | Q8_0 | 441.9tok/s | 39,673.0tok/s | 79.4ms | r_b948ehgi8g4 |
| concurrent-decode | 1.2B | RTX 5880 (48GB) + AMD EPYC 9T24 96-Core Processor (16c) + 62GB | ollama@0.34.2 | Q8_0 | 464.2tok/s | no data | no data | r_b948ehgi8g4 |
| agent-trace | 1.2B | RTX 5880 (48GB) + AMD EPYC 9T24 96-Core Processor (16c) + 62GB | ollama@0.34.2 | Q8_0 | 452.3tok/s | 91,077.2tok/s | 21.8ms | r_b948ehgi8g4 |