The only way to stop is to crash; recovery is tested by every deploy. Separate shutdown code goes unexecuted for years and therefore doesn’t work.
The source is George Candea and Armando Fox, “Crash-Only Software” (HotOS IX, 2003). Their observation: a system has two ways to stop — clean shutdown and crash — and must survive the second anyway, because power cuts and kernel panics do not ask. If it must recover from a crash, a separate clean-shutdown path is a second, rarely run, code path to the same place. Make crash the only stop and start the only recovery, and the recovery code runs on every restart, so its bugs are found in daylight. Their requirements for that: all important state lives in dedicated crash-safe stores (a database with a write-ahead log, not process memory), components are restartable on their own, requests carry timeouts and are retried — which needs idempotent operations (22). Erlang’s “let it crash” with supervisor trees is the same idea at the process level.
// The orchestrator (k8s) kills pods constantly — so startup must be recovery
func main() {
state := mustRecoverFromLog(walPath) // startup == recovery, always
// graceful shutdown is minimal: just release leadership / drain the LB;
// correctness does NOT depend on it
}A stream consumer example: it commits its offset only after the effect is durable, so a SIGKILL at any instruction means some messages are processed again on restart, never lost — and processing again is safe because the sink is idempotent. A test that kills the process at random points under load (77) checks the property directly.
A clarification. The thesis is that correctness must not depend on graceful shutdown. A clean shutdown in itself remains a useful optimization: draining connections and the load balancer, handing off leadership, reducing rework after a restart. Dropping it is not a consequence of crash-only but a separate decision. The cost is recovery time: a large log replayed on every start makes deploys slow, so checkpoints bound it.