MI300X LLM benchmark
No signed MI300X submissions yet. Check the configuration guidance below, or contribute a reproducible measurement from your own machine.
Specs vendor specs, not measured
- VRAM
- 192 GB
Plan memory before choosing a model
The catalog lists 192 GB of VRAM for this hardware entry. Check the capacity of your actual card. Model weights, context cache and runtime overhead must share the available memory. Capacity alone does not establish which model will fit or how fast it will run. Use the memory-fit calculator for your model and context as an estimate, then verify with the intended runtime. No signed MI300X measurement is available yet.
Contribute a signed MI300X benchmark
Run on YOUR hardware to populate this page: pipx install llm-speed && llm-speed bench
$ pipx install llm-speed && llm-speed bench
Community folklore on MI300X
2 unverified claims extracted from Reddit/HN comments. Lower trust than signed runs above; every row links to the source.
- communityconfidence 55%
353.0tok/s — llama-2-70b on MI300X FP16
our signed data: MI300X · llama-2-70b
“85c28e456902a874152d145e2ea3f4d28f84ebf8 After some calculations: MI300X: * Price: $15,000 (or 1.5 million alpaca tokens) * Performance: 353 tokens/s/GPU (FP16) * Memory: 192GB HBM3 (that's a lot of context for your LLM to chew on) vs H100 * Bandwidth: 5.2 TB/s (faster than yo…”
- communityconfidence 45%
370.0tok/s — on MI300X FP16
our signed data: MI300X
“on) vs H100 * Bandwidth: 5.2 TB/s (faster than your desk llama can spit) H100: * Price: $28,000 (approximately one kidney) * Performance: 370 tokens/s/GPU (FP16), but it doesn't fit into one. * Memory: 80GB (MI300X has almost 2.5 times more VRAM!!) Key Points: 1. H100 is \~4.…”
Request a run on MI300X
No measured run yet for these pairs. Vote for the one you want; the most-requested runs go first. See all requests.
We never sell or share your email. Privacy