When we hit 40,000 requests per second last October, our auth service became the bottleneck nobody wanted to own. Every API call from São Paulo or Singapore had to round-trip to us-east-1 just to validate a JWT. The latency tax was predictable — roughly 180 to 220ms added per request — but as we expanded into APAC, it started to feel less like a tax and more like a toll gate.
The Centralized Bottleneck
I spent two weeks last winter profiling our request path end to end. The math was brutal: on a typical edge-to-origin flow, token validation consumed 40% of our total latency budget. Our security team was reluctant to decentralize — they worried about cache invalidation and key rotation across 300 locations. But when our p99 in Sydney crossed 800ms, the conversation changed overnight.