How is AI API latency measured on llmlatency.dev?

Probes in four regions (Europe, US, Asia, South America) connect directly to each provider's official API host and measure DNS, TCP, TLS and time-to-first-byte for edge latency, and time-to-first-token for inference latency, on a repeating schedule. Figures are p50/p95 over the last 24 hours.

Is the latency measured through a gateway like OpenRouter?

No. Each provider's own API endpoint is measured directly, so the numbers reflect that provider's infrastructure rather than an aggregator's.

What do p50 and p95 mean here?

p50 is the median (a typical request); p95 is the 95th-percentile slow tail. Both are nearest-rank percentiles over the last 24 hours of probes.

Methodology — how AI API latency is measured

Updated July 24, 2026

Latency here is measured directly from probes in four regions — not scraped from pricing pages and not routed through an aggregator gateway. This page documents exactly how, so the numbers can be audited and cited with confidence.

What is measured

Two independent metrics per provider:

Edge latency (time-to-first-byte, TTFB) — a probe connects to the provider's official API host and measures DNS resolution, TCP handshake, TLS handshake and time to the first response byte. This isolates network and edge performance from model compute.
Inference latency (time-to-first-token, TTFT) — the time from request to the first streamed token of a real streamed completion. This needs an API key per provider, so it is measured only where we hold one and appears as a separate table on those region pages. Coverage is deliberately partial: every headline ranking on this site is edge latency (TTFB), never a mix. We publish nothing rather than estimate a provider we cannot actually call.

Where it is measured from

Independent probe nodes in four regions, so latency reflects real geographic distance to each provider's infrastructure: Europe (Germany), US (Central), Asia (Tokyo) and South America (São Paulo). Rankings routinely differ by region — the fastest API from Europe is often not the fastest from Asia. That per-region difference is the reason this site exists.

How often

Each provider is probed on a repeating schedule (every few minutes). Pages rebuild automatically and show a visible “Updated” / “Last verified” date. Reported figures are percentiles over the last 24 hours.

Percentiles, not averages

We publish p50 (median — a typical request) and p95 (95th percentile — the slow tail users actually feel), plus uptime (share of probes returning a successful response). Percentiles are nearest-rank; averages are avoided because a few outliers distort them.

Speed Index

The Speed Index is a composite 0–100 score for comparing providers across regions at a glance. In each region a provider scores fastest_p50 / provider_p50 × 100 — so the region’s fastest API scores 100 and one twice as slow scores 50 — and the score is averaged over the regions where the provider is measured. The homepage lists only providers measured in all regions, so the comparison is like-for-like. It is derived entirely from the measured p50 figures above: reproducible, not a subjective rating.

What this is — and isn’t

It is an independent, provider-neutral, reproducible signal of how quickly each API responds by region, over time.
It is not a benchmark of model quality, throughput (tokens/sec) or price. Measured latency depends on network path and endpoint behaviour and can change without notice.
Every number is measured by this system; nothing is estimated or copied from a provider’s marketing.

Use the data

All rankings are free (CC-BY-4.0). Machine-readable: JSON API, per-page markdown (append .md or send Accept: text/markdown), full corpus, and an MCP endpoint for agents.