Incident Library?

Incident Library?

Component ?

GKE control plane GKE 컨트롤 플레인 · `gke-control-plane`

2 cases · 2019

Root cause ?Cloud provider outage 1 Configuration change 1

Contributing factor ?Excessive blast radius 1 Unreviewed defaults 1 No alert on limit 1 No health probes 1 No safeguard on destructive change 1 Unrepresentative staging 1

Symptom ?Service outage 2 Upgrade stall 2 Autoscaler misbehaviour 1 Control plane unresponsive 1 Node recreation loop 1

Cases, by severity

IL-0008 A validating webhook with a dead backend stalled a GKE master upgrade — Jetstack, 2019, sev1
Root cause Configuration change
IL-0014 A GKE upgrade that stalled, left zombie nodes and broke an ingress — loveholidays, 2019
Root cause Cloud provider outage

View on the graph