Timeout + retry with jitter, circuit breaker, bulkhead

pattern
established
fragment

Scope. Timeout, retry with jitter, circuit breaker and bulkhead are not interchangeable — they cover different failures: a response that is too slow, a one-off error, sustained degradation of a dependency, and failure spreading across pools. None of them replaces queue bounds and load shedding (23); a retry without a budget and an end-to-end deadline amplifies overload instead of easing it.

Retry without jitter is a distributed metronome: a synchronized crowd finishes off a recovering service.

fragment

Rust · full jitter (AWS recipe)

async fn retry<T, E>(mut f: impl FnMut() -> BoxFuture<'static, Result<T, E>>) -> Result<T, E> {
    let base = Duration::from_millis(100);
    for attempt in 0..5 {
        match f().await {
            Ok(v) => return Ok(v),
            Err(_) if attempt < 4 => {
                let cap = base * 2u32.pow(attempt);
                sleep(Duration::from_millis(rand::random_range(0..cap.as_millis() as u64))).await;
            }
            Err(e) => return Err(e),
        }
    }
    unreachable!()
}

Go · end-to-end deadline

// the context deadline is end-to-end; the caller sets the timeout, not the library
ctx, cancel := context.WithTimeout(ctx, 800*time.Millisecond)
defer cancel()

resp, err := s.gw.Charge(ctx, req) // the whole stack below honors the deadline

Java · Resilience4j

// breaker + bulkhead by composing decorators
var decorated = Decorators.ofSupplier(() -> gateway.charge(req))
    .withCircuitBreaker(CircuitBreaker.ofDefaults("gw"))
    .withBulkhead(Bulkhead.of("gw", BulkheadConfig.custom().maxConcurrentCalls(20).build()))
    .withRetry(Retry.of("gw", RetryConfig.custom().intervalFunction(
        IntervalFunction.ofExponentialRandomBackoff()).build()))
    .decorate();

Python · tenacity

# exponential backoff + jitter, declared declaratively
from tenacity import retry, stop_after_attempt, wait_random_exponential

@retry(stop=stop_after_attempt(5), wait=wait_random_exponential(multiplier=0.1, max=3))
async def charge(req: PaymentReq) -> Payment: ...

What the example above is missing. It retries every error. A production policy distinguishes retryable errors (timeout, 5xx, overload) from non-retryable ones (validation, authentication, 4xx) — retrying the latter is pointless and harmful; honors Retry-After instead of its own backoff; is bounded by an end-to-end deadline, not by a sum of individual timeouts; is cancelled together with the context; has a per-client retry budget so that retries don’t make up most of the traffic; and never retries a non-idempotent operation without an idempotency key (22). Attempt counts and budget exhaustion are metrics; otherwise degradation is invisible.