The llm-speed dataset

Current dataset: version 2026-10-17.3

522 measured workload results from 277 public runs · 71 models · 28 hardware configurations · runtimes llama.cpp, mlx, ollama. One row per measured result; every row links to its run page. Runs flagged for review are excluded. No submitter keys, fingerprints or handles are included.

Versions are dated and never overwritten; the newest version is listed in /data/open-dataset.json. License: CC BY 4.0.

Fields

run_id

Run ID; the run page is /r/<run_id>

run_url

Run page URL

date

Date the run was received (UTC, YYYY-MM-DD)

hardware

Hardware summary as reported by the benchmark CLI (accelerator + CPU + RAM)

model

Model name as reported by the runtime

model_size

Parameter count as reported by the runtime, if any

quant

Quantization as reported by the runtime, if any

runtime

Inference runtime (backend), e.g. llama.cpp, ollama, mlx

runtime_version

Runtime version, if reported

workload

Suite workload, e.g. chat-short, agent-trace

suite_version

Benchmark suite version

decode_tps

Measured decode speed, tokens/second

prefill_tps

Measured prompt-processing speed, tokens/second

ttft_ms

Measured time to first token, ms

prompt_tokens

Prompt tokens

output_tokens

Generated tokens

batch_size

Concurrent sequences

label

Always 'measured': every row is a signed measurement

How to cite

llm-speed. Measured LLM inference benchmarks, version 2026-10-17.3. https://llm-speed.com/data. CC BY 4.0.

Archived July 8 summary snapshot

The July snapshot is also published as a HuggingFace dataset (with a live Dataset Viewer) and archived on Zenodo with a citable DOI, for notebooks, training pipelines, and papers.

Prefer more recent submissions? The JSON API returns a bounded list of run summaries (50 by default); it is not a full-corpus download. Per-run detail is at api.llm-speed.com/v1/results/{id}. Every run also has a citable page at llm-speed.com/r/{id}.

Reuse it (attribution required)

The data is licensed Creative Commons Attribution 4.0. Use it anywhere, including commercially, as long as you credit llm-speed with a link back. Copy one of these:

Data from llm-speed (https://llm-speed.com/data), CC BY 4.0.

<a href="https://llm-speed.com/data">Benchmark data by llm-speed</a>, CC BY 4.0.

Building a tool on this data (a VRAM calculator, a price-per-token comparison, a hardware guide)? That is exactly what the license is for. A link back is all we ask.

llm-speed. llm-speed: signed LLM inference-speed benchmarks. July 8, 2026 snapshot. Zenodo (2026). https://doi.org/10.5281/zenodo.21254813. CC BY 4.0.

How to interpret the data

These are submitted measurements, with varying artifacts, workloads and runtime settings. A signature preserves submitted provenance; it does not independently verify hardware, answer quality or exact reproducibility. The bulk summaries omit signatures and per-workload detail. Open a source permalink before comparing rates, and preserve missing fields instead of inferring a configuration. The focused study above documents its exact cohort and limitations separately.