Privacy by design

Retention and deletion are a property of the schema, not a script. The direct conflict with event sourcing is resolved by crypto-shredding: PII is encrypted with a per-subject key; deleting the subject = deleting the key, and the log stays immutable.

The term comes from Ann Cavoukian, then Information and Privacy Commissioner of Ontario, whose “Privacy by Design: The 7 Foundational Principles” (2009) include privacy as the default setting and privacy embedded into design rather than bolted on. The EU’s GDPR made a version of it law: Article 25, “data protection by design and by default”, alongside data minimisation and storage limitation in Article 5 and the right to erasure in Article 17. For an engineer the translation is concrete: each personal field has a declared purpose and retention period, collection is the minimum the purpose needs, and deletion is a tested code path that runs on a schedule, not a ticket someone handles by hand.

record CustomerEvent(UUID customerId, EncryptedPayload pii, byte[] nonPii) {}

// keystore.delete(customerId) => all the subject's events unreadable forever,
// the Kafka log is not rewritten

Why crypto-shredding: an event log or a compacted topic is designed never to be rewritten, and backups are deliberately hard to edit. Encrypting each subject’s personal fields with their own key moves deletion from the many places the data lives to the one place the key lives.

What crypto-shredding does not do. Deleting the key makes exactly the encrypted payload inaccessible — and only if every copy of the data and keys falls under the same deletion protocol. What usually slips past it: PII in logs, traces and DLQs; derived tables and search indexes; caches; exports; key backups; decrypted copies in consumers; aggregates that allow re-identification. Crypto-shredding is part of a deletion strategy, not a substitute for a data inventory and retention enforcement. Whether encrypted data with a destroyed key counts as erased is a legal judgement, not an engineering one — ask before relying on it.