Expiry of a hot key = a synchronized volley from all its subscribers into the origin, a self-DDoS on a schedule. Three mandatory mechanisms: collapse concurrent misses into one request (singleflight), randomize TTLs (so keys don't expire in unison), and cache absence (otherwise every request for a nonexistent id hits the database).
var g singleflight.Group
func (s *Svc) Product(ctx context.Context, who Principal, id string) (Product, error) {
// one key for cache AND singleflight: tenant + access scope + id,
// or a caller joins a flight that loaded someone else's view
key := who.TenantID + "|" + who.Scope + "|" + id
if p, ok := s.cache.Get(key); ok { return p, nil }
v, err, _ := g.Do(key, func() (any, error) { // N misses → 1 origin request
p, err := s.db.Product(ctx, who, id)
if errors.Is(err, ErrNotFound) {
s.cache.SetNegative(key, 30*time.Second) // negative cache, short TTL
return Product{}, err
}
s.cache.Set(key, p, jitterTTL(5*time.Minute)) // ±20% — desynchronizes expiry
return p, err
})
...
}# probabilistic early refresh: hot keys are refreshed BEFORE they expire,
# with probability rising as the TTL approaches
import math, random, time
def should_refresh(expires_at: float, delta: float, beta: float = 1.0) -> bool:
return time.time() - delta * beta * math.log(random.random()) >= expires_atNegative caching without a short TTL is a way to cache a transient error forever; cache absence, not errors (an origin 5xx is never written to the cache).
Limits of applicability. Negative caching is needed where misses are genuinely expensive; otherwise it is an extra layer. The cache key must include the tenant and access rights — otherwise singleflight will merge requests from users with different visibility and hand out someone else's data. And cancelling the first caller's context must not cancel the work for everyone who joined it.