glm-4.7-flash
12 workload results across 2 hardware configurations.
Fastest local config
164.9 decode tok/s
on RTX 4090 (24GB) + AMD EPYC 75F3 32-Core Processor (64c) + 504GB via ollama (Q4_K_M). see full run
For a standard Q4_K_M quantization using Ollama, here are measured results for short-chat generation.
On Ollama Q4_K_M, glm-4.7-flash (29.9B) is fastest on the RTX 4090 at 164.9 tok/s, 23% faster than the RTX 3090 (134.6 tok/s).
| Card | Decode tok/s | TTFT (105-token prompt) | Run |
|---|---|---|---|
| RTX 4090 | 164.9 (single run) | 28106 ms | r_ted0u8uyre0 |
| RTX 3090 | 134.6 (single run) | 24309 ms | r_kalwcyab-86 |
glm-4.7-flash (29.9B), Q4_K_M, Ollama, chat-short, batch 1. Only cards with their standard memory are listed; CPU-only hosts are excluded.
Local runs (12 workload results)
Runs from contributors' own machines via MLX, llama.cpp, vLLM, exllamav2, or ollama. Signed on the submitter's hardware.
RTX 4090 (24GB) + AMD EPYC 75F3 32-Core Processor (64c) + 504GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 164.9tok/s | 3.74tok/s | 28,106ms | r_ted0u8uyre0 |
| chat-long | ollama@0.31.1 | Q4_K_M | 146.1tok/s | 3,617.7tok/s | 866ms | r_ted0u8uyre0 |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 155.2tok/s | no data | no data | r_ted0u8uyre0 |
| agent-trace | ollama@0.31.1 | Q4_K_M | 155.5tok/s | 3,224.6tok/s | 641ms | r_ted0u8uyre0 |
| chat-short | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_o636l3cc-rr |
| chat-long | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_o636l3cc-rr |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | no data | no data | no data | r_o636l3cc-rr |
| agent-trace | ollama@0.31.1 | Q4_K_M | 129.9tok/s | 2,491.1tok/s | 1,262ms | r_o636l3cc-rr |
RTX 3090 (24GB) + AMD Ryzen Threadripper PRO 5975WX 32-Cores (32c) + 504GB
| Workload | Backend | Quant | decode tok/s | prefill tok/s | TTFT | Run |
|---|---|---|---|---|---|---|
| chat-short | ollama@0.31.1 | Q4_K_M | 134.6tok/s | 4.32tok/s | 24,309ms | r_kalwcyab-86 |
| chat-long | ollama@0.31.1 | Q4_K_M | 116.1tok/s | 2,090.1tok/s | 1,499ms | r_kalwcyab-86 |
| concurrent-decode | ollama@0.31.1 | Q4_K_M | 133.6tok/s | no data | no data | r_kalwcyab-86 |
| agent-trace | ollama@0.31.1 | Q4_K_M | 125.2tok/s | 2,901.4tok/s | 614ms | r_kalwcyab-86 |