IL-0022 Anthropic · 2025-08-05ai-infra
Three unrelated problems overlapped. Some requests were routed to servers set up for the long-context pool, and sticky sessions kept follow-ups on the wrong server; a later load-balancing change raised the share of affected traffic. A misconfiguration on the TPU servers corrupted token selection. And a latent compiler bug in the approximate top-k operation surfaced when an earlier workaround was removed. Each showed differently on each platform, standard benchmarks did not catch them, and privacy controls limited what engineers could inspect. Fixes were a routing correction, two rollbacks, and exact top-k at higher precision.
Users reported worse answers from early August, at first indistinguishable from ordinary feedback noise. By late August the reports were persistent and some responses contained characters from the wrong language or syntax errors.
Root cause ? Configuration change
Trigger A routing change on 5 August, a load-balancing change on 29 August that widened its effect, and two server-side deployments on 25 and 26 August.
Rising user reports, including public ones; the spike was not connected to the load-balancing change right away.
Benchmarks and canaries did not catch the degradation, noisy evaluations and limited access to user interactions slowed diagnosis, and continuous quality checks with user feedback channels are needed.
–