IL-0201 GitHub · 2025-03-17kubernetes
Between 18:05 UTC on March 17 and 09:50 UTC on March 18, 2025, GitHub.com had intermittent failures in web and API requests. GitHub names as the root cause a framework upgrade that caused kernel panics in its Kubernetes infrastructure. During the incident it attributed the errors to problems with restarts in that infrastructure; requests saw HTTP 500 responses and timeouts, mostly for issues and pull requests. The peak error rate was 0.165% of all requests.
GitHub mitigated the incident by downgrading the framework, and kept it downgraded until it could turn off the feature that caused the problem. The write-up names neither the framework nor the feature.
From 18:05 UTC on March 17, 2025, GitHub.com web and API requests failed now and then for a small percentage of users, mostly requests about issues and pull requests. At 18:39 UTC the status page reported degraded performance for Issues.
Later updates told users they might get intermittent HTTP 500 responses from Issues and that a retry of the same request might succeed. GitHub described the cause at that point as problems with restarts in its Kubernetes infrastructure. At the peak, over a 15-minute window, 0.165% of requests to the affected API surfaces timed out or returned errors; afterwards the impact was minimal while the investigation continued.
Symptom ? High error rateIntermittent timeouts
Root cause ? Breaking change in an upgrade
The status page opened the incident at 18:39 UTC, 34 minutes after the failures began, as an investigation of reports of degraded performance for Issues.
–
same root cause · shared Linux kernel