109

Competing explanations

#

A true claim can still fail to explain your case. Before acting on an explanation, hold its rivals and find the observation that tells them apart.

Bills fell after the lamps were replaced. “Because of the lamps” is one explanation; a closed wing, a new tariff, a warm winter and shorter hours explain the same fact, and several may act at once. Asked “why did costs fall after the lamps?”, a model gives the one explanation the question already contains, and gives it confidently. Checking that LED lamps use less power does not tell you what share of the drop they caused. Ask for alternatives explicitly, write down what each predicts differently, and look for the observation that separates them.

The method is old. The geologist T. C. Chamberlin, “The Method of Multiple Working Hypotheses” (Science, 1890), warned that a single favoured hypothesis becomes a “ruling theory” that bends the evidence toward itself, and recommended holding several at once. John Platt, “Strong Inference” (Science, 1964), turned it into a procedure: list the alternatives, design the experiment whose outcome excludes some of them, run it, repeat. Both are about scientific practice; neither measured how much better it works, so treat them as well-argued method, not proven effect.

An incident version. p99 latency doubled after Tuesday’s deploy. Rivals: the new code; a dependency that degraded the same hour; a traffic mix shift toward a heavy tenant; a noisy neighbour on the node pool. They predict different things: the code predicts the regression follows the new version across hosts and tenants; the dependency predicts it shows in that dependency’s span on old and new versions alike; the tenant predicts it concentrates in one customer. One query over wide events (28) grouped by version, tenant and node separates them — rolling back first and seeing the problem stay would separate them too, at higher cost.

In systems this is incident analysis: a postmortem with one root cause usually stopped too early. Observability exists to answer questions that separate hypotheses. The cost is time: under a live outage, mitigate first on the most likely cause, then discriminate.

Last technical review: 2026-08-04 · Reference · 112 principles