About llm-speed

llm-speed is a community benchmark for LLM inference speed. One CLI, one methodology, real numbers across hosted APIs, consumer GPUs, Apple Silicon, and prosumer rigs.

Why this exists

Artificial Analysis covers hosted APIs and datacenter accelerators; MLPerf is enterprise rigs; r/LocalLLaMA is folklore in comment threads. Nobody owns the union — consumer-local plus hosted-API numbers under one protocol, with a permalink per result. That's what this is.

Who runs it

One maintainer, publishing under the pseudonym meadow-kun. Solo project with an open-source CLI, no company behind it. Issues and DMs welcome via GitHub. What you get for trusting me with a submission: every run is signed by an Ed25519 keypair on your machine, the canonical bytes are auditable from the run page, the suite version is pinned, and the public CLI is Apache-2.0 in its linked repository. The website and ingestion service are maintained separately. Signing preserves provenance; it does not independently verify hardware or make different configurations comparable. Disputes happen in public GitHub issues against the run id. Numbers belong to their submitters, not to me.

How it works

• Install the CLI. It detects your hardware and the inference backends you have available.
• Run the workload suite. You get tok/s, TTFT, and latency numbers in about a minute.
• Each result is signed by an Ed25519 keypair on your machine and posted to a public, permalinked run page.
• Read the methodology for the full workload spec and how disputes work.

Related work

These are the resources we've learned from and continue to point people toward:

• Artificial Analysis — hosted-API and datacenter accelerator benchmarks.
• LocalScore — open local-LLM benchmarks (Mozilla Builders).
• r/LocalLLaMA — the community where most of this knowledge lives.

Open source

The CLI source is public under Apache-2.0. The methodologydocuments the measurements and their limitations. The benchmark data is CC BY 4.0 — reuse it freely, including commercially, with attribution (a link back to llm-speed.com or the specific run). Submit issues or ideas via GitHub. Results belong to the people who submitted them.