IL-0124 OpenAI · 2026-03-04ai-infra
OpenAI allocates compute capacity to its models through automated infrastructure management systems. A protective circuit breaker had paused certain scheduling operations. While the pause lasted, the automated capacity management systems kept queuing adjustments, and those adjustments were based on system state that had since become outdated.
When the circuit breaker was released, the queued changes all ran at once. The batch of capacity reassignments removed multiple inference engines from service at the same time, which reduced available capacity, and API error rates and latency rose for about 30 minutes. Engineers identified the capacity imbalance and reassigned infrastructure to the affected models by hand; as engines came back online, error rates returned to normal by 4:56 PM PST.
Between about 4:26 PM and 4:56 PM PST on 4 March 2026, some OpenAI API requests to several models failed or took longer than usual, and some Codex traffic was affected too. The overall API success rate dropped temporarily.
The impact differed by model. Affected models included gpt-4.1, gpt-4.1-mini, gpt-4o-mini, the gpt-5 series, OpenAI o3 and the Codex models; GPT-4o mini was the most affected major model.
Symptom ? High error rateHigh latency
Root cause ? Software defect
–
Pausing one automated system does not stop the systems that feed it. Changes queued during the pause reflect the state at the time they were queued, and releasing the queue all at once applies those stale decisions together. The fixes OpenAI names are detecting state drift, making circuit breaker status visible, and preventing large batches from executing after a long pause.
–