Glossary — AI API latency terms
Plain-language definitions of the latency and reliability terms used across this site.
- Time to first byte (TTFB)
- The time from starting a request to receiving the first byte of the response, including DNS, TCP and TLS setup. Here it is the primary edge-latency metric and isolates network and endpoint speed from model compute.
- Time to first token (TTFT)
- For a streamed AI completion, the time from sending the request to the first token arriving. It captures queueing and model start-up, not just the network.
- p50 (median)
- The middle value: half of measured requests were faster and half slower. Represents a typical request.
- p95
- The 95th percentile: only 5% of requests were slower. Captures the slow tail that users notice, which averages hide.
- Uptime
- The share of probes that returned a successful response over the measurement window, expressed as a percentage.
- Edge latency
- Latency attributable to the network path and the provider's connection termination (DNS/TCP/TLS/first byte), as opposed to model inference time.
- Cold start
- Extra latency when an endpoint or model must be initialised because it was idle, versus a warm path already serving traffic.
- Region
- The geographic location a measurement was taken from. Latency depends on distance to the provider's infrastructure, so rankings differ by region.
- Speed Index
- A composite 0-100 score comparing providers across regions: in each region, fastest_p50 / provider_p50 x 100 (100 = as fast as the region leader), averaged over measured regions. Derived from measured p50, not a subjective rating.