A sound implementation does not ask users to refresh until it works. Each phase has an input, an output, a timeout, and a terminal state. Liveness asks whether restart helps; readiness asks whether new traffic is safe. Putting a slow database in liveness restarts every instance and amplifies the outage.
A release must be reproducible, observable, and reversible, including configuration and data compatibility. A health check proves process life; critical routes, dependencies, and background work need separate gates.
Define the system contract first
This capability crosses clients, networks, and servers, so a local optimization can create a system failure. Decisions must constrain both endpoints, persisted truth, and operating budgets together.
- Startup shields liveness during migration or warmup; readiness checks admission state and critical dependency budget; liveness detects only unrecoverable deadlock.
- Give state one owner, a version, and terminal states; callbacks may mutate only the version that created them.
- Ship conservative defaults, server-side ceilings, and a rollout switch instead of trusting browser-provided numbers as resource budgets.
The delivery standard for Liveness, Readiness, and Startup Probes Have Different Jobs is a usable normal path, convergent failures, bounded resources, and a state users can understand. The result is a production capability that can be explained, degraded safely, and rolled back—not a demo that works once.
What can break that contract
Failure and success must share one state model. An error toast that neither releases resources nor propagates a terminal state leaves dirty work for the next recovery attempt.
- Expensive probe queries become load, while synchronized not-ready transitions at one timeout can overload surviving nodes.
- Fixing only the UI leaves queues, locks, or expired credentials for the next operation to inherit and fail again.
- Without backpressure or quota, a slow consumer raises memory, queue depth, and tail latency until unrelated users are affected.
How to test the contract line by line
Do not stop verification when the final action succeeds. Count side effects, measure wait time, inspect privacy, and prove the next run begins from a clean baseline.
- Inject slow startup, a two-second database blip, event-loop deadlock, read-only disk, and drain; only the intended probe changes without orchestrator amplification.
- Run one hundred start, fail, retry, and cancel cycles; handles, listeners, queues, and temporary data must return to baseline.
- Use fault injection to prove alerts precede user reports and operators can locate the failing phase from bounded evidence.
Completion is not one passing path. Every terminal state reconciles, automation stays below user intent, and every operational cost has an explicit ceiling.