The Operator's Guide to Load Testing

By Marcus Hale · August 6, 2026 · Engineering

Load tests fail to predict production for a consistent reason: the traffic shape is wrong. Uniform random requests against one endpoint tell you about that endpoint. The production killer is the burst of clients retrying in sync after a thirty-second blip, or the cache-cold crawl at 5 a.m. when the SEO bot wakes up.

Recording and replaying the real distribution - endpoint mix, think times, header entropy - is worth more than doubling the request rate of a synthetic profile. Pay particular attention to the arrival process: production arrivals are bursty, not Poisson, and queues that look fine at constant rate fall over under realistic variance.

Finally, keep the test close to continuous. A load test that runs once a quarter answers what the system did last quarter. Wire a smaller version into the pipeline and you will catch the slow leaks before your customers do.