Property-based / type-driven testing

An example test checks the cases you thought of; a property test checks a claim that must hold for every input, and a generator hunts for the counterexample — and, once it finds one, shrinks it to a minimal one. Properties that pay off almost everywhere: roundtrip (parsing what was written yields the original), conservation (the sum does not change), agreement with a naive reference implementation. Types are the same idea one step earlier: what the compiler guarantees (3) no longer needs testing.

The method comes from Koen Claessen and John Hughes, “QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs” (ICFP 2000): the programmer states properties, the library generates random inputs, and a failing input is shrunk to a small counterexample. Hypothesis (Python), jqwik (Java) and proptest (Rust) descend from it. Coverage-guided fuzzing (AFL, libFuzzer, Go’s built-in fuzzing) is the same loop with a different generator: instead of a distribution you describe, it mutates inputs that reach new code paths.

// a property instead of examples
@Property
void settlementConservesMoney(@ForAll("trades") List<Trade> trades) {
    var entries = Settlement.settle(trades, RATES, DAY);
    assertThat(sumDebits(entries)).isEqualTo(sumCredits(entries));
}

// roundtrip: the cheapest property, and the one that catches the most
proptest! {
    #[test]
    fn serde_roundtrip(cfg in any::<Config>()) {
        let json = serde_json::to_string(&cfg).unwrap();
        prop_assert_eq!(serde_json::from_str::<Config>(&json).unwrap(), cfg);
    }
}

// testing/quick is obsolete; the idiom is rapid or built-in fuzzing
func FuzzParseEmail(f *testing.F) {
    f.Fuzz(func(t *testing.T, raw string) {
        e, err := ParseEmail(raw)
        if err == nil && !strings.Contains(e.String(), "@") {
            t.Fatalf("parsed invalid email: %q", raw) // the parser accepted garbage
        }
    })
}

from hypothesis import given, strategies as st

@given(st.lists(trade_strategy()))
def test_settlement_conserves_money(trades):
    entries = settle(trades, RATES, DAY)
    assert sum_debits(entries) == sum_credits(entries)

The strongest form for backend code is a model. A stateful property test runs random sequences of operations — put, delete, compact, restart — against the real component and against a trivial model (a dict), and asserts after every step that they agree. That is how bugs that need five specific operations in a row get found: nobody writes that example by hand. Hypothesis ships this as RuleBasedStateMachine; the same idea at cluster scale is checking histories against a model (98).

Limits. A property test samples; it is not a proof, and a generator that never produces the dangerous shape (empty lists, duplicate keys, Unicode, boundary timestamps) passes forever. Weak properties (“does not throw”) catch little; the work is in finding a property that is strong and still easy to state, and for some domains — rendering, ranking, anything judged by taste — there is none, and examples remain the right tool. Pin the seed and keep found counterexamples as regression cases, or a failure seen once in CI will not come back on demand.