IL-0187 Yonghyun Kim · 2025-10-11 (published)kubernetes
At the author's previous company, business applications on Kubernetes embedded the Spring Cloud Config Client library to load configuration from a Spring Cloud Config Server, and in one Kubernetes environment a liveness probe was configured by default. The Spring Boot actuator health endpoint used by the probe automatically included the config server's state once the Config Client library was present.
When the config server was taken down briefly for an upgrade, every client's health check failed because of that external dependency. Kubernetes restarted the pods, they failed again, and the applications fell into a restart loop that took down an affiliate's services, although nothing was wrong with the applications.
The author's conclusion is that a liveness probe must check only the application itself. Readiness probes should handle temporary problems such as load, network loss or an unavailable config server by taking the pod out of traffic. If a liveness probe is needed, external health indicators such as DataSourceHealthIndicator and ConfigServerHealthIndicator should be excluded or a separate path such as /healthz used, with a generous initialDelaySeconds and a long failure window, for example failureThreshold: 10 and periodSeconds: 6 for 60 seconds of continuous failure.
While the Spring Cloud Config Server was briefly down for an upgrade, every client application began to answer its liveness probe with a failure saying it could not reach the config server.
Kubernetes treated the failed health check as a dead application and killed and restarted the pods. The applications themselves were fine, and the config server was about to come back, but the pods kept being killed and restarted without a chance to recover, and the services of one affiliate company went down.
Root cause ? Dependency failure
–
–
–
A liveness probe that includes an external dependency turns a short outage of that dependency into a restart loop of every client. Keep external dependencies out of liveness checks and handle temporary problems with readiness probes.
Libraries can add health indicators to Spring Boot's actuator health endpoint automatically, so check what the probe path actually reports. The author ran another Kubernetes environment for two years without liveness probes, relying on strict readiness probes and on the application terminating itself on unrecoverable faults.
–