Schema compatibility as a type

A schema shared by a writer and a reader that deploy separately is a contract with two directions: new readers must read old data (backward compatibility) and old readers must survive new data (forward compatibility). Protobuf and Avro make the safe moves explicit: add, never reuse a field number, reserved for the deleted ones — and a registry turns those rules into a check that fails the build.

All three rules follow from one fact: on the wire a field is identified by its number, not its name — the name lives only in the .proto and in generated code. Hence renaming a field is safe (same number), changing its number is equivalent to replacing the field, and an old reader skips an unfamiliar number as an unknown field — which is why adding is compatible. Hence also the worst case: if a deleted field’s number is given to a new field of a different type, an old reader will parse it with the old schema and silently get someone else’s data — not an error, but plausible garbage. reserved exists precisely so the compiler won’t let you do that; it is the same burned identifier as a principle’s retired number in this reference.

message Order {
  reserved 3, 7;                 // deleted fields: numbers burned forever
  reserved "legacy_status";

  string id = 1;
  int64 amount_cents = 2;
  OrderStatus status = 4;        // adding is the only safe operation
}

// a consumer must survive unknown fields (forward compat)
ObjectMapper mapper = JsonMapper.builder()
    .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) // deliberately
    .build();

#[derive(Deserialize)]
#[serde(deny_unknown_fields)]  // at a TRUST BOUNDARY: strictness;
struct AdminCommand { ... }    // on a service-to-service contract: the opposite,
                               // tolerance of new fields

Where the rules come from. They are not folklore but the formats’ own specifications: the Protocol Buffers language guide (“Updating a message type”) and the Avro specification’s “Schema Resolution” section, which defines exactly how a reader’s schema is matched against a writer’s. Avro has no field numbers — fields are matched by name — so its version of the rule is different: a field added for a new reader needs a default, or old records cannot be read. Confluent Schema Registry encodes the directions as compatibility modes (BACKWARD, FORWARD, FULL, and _TRANSITIVE variants that check against every earlier version, not just the last); the mode follows from who upgrades first — consumers first is backward, producers first is forward.

What a compatibility check cannot see. It checks shape, not meaning. Changing amount from cents to euros, or a timestamp from UTC to local time, passes every registry check and corrupts every reader; such a change is a new field with a new name, migrated with expand — migrate — contract (17). Some changes are wire-compatible and still lossy: widening int32 to int64 parses on both sides, but an old reader truncates large values. And the tolerance for unknown fields shown above belongs on service-to-service contracts; at a trust boundary, strictness wins (19, 32).