SOURCE B: Retry Strategies, Timeouts, and Idempotency

Retries are the first line of defence against transient failures but are dangerous without careful design. Naive immediate retries under load amplify traffic at exactly the wrong moment — the thundering herd problem. Exponential backoff with jitter is the standard remedy: each attempt waits 2^n seconds plus a random jitter value, spreading retry storms across a window. AWS, Google, and most cloud SDKs implement this by default; teams writing custom HTTP clients must implement it explicitly.

Timeouts are a prerequisite for effective retries. A service that waits indefinitely for a response will exhaust its thread pool before retrying becomes possible. Every outbound network call should carry an explicit deadline. The challenge is setting the right value: too short and you see spurious timeouts under GC pauses or network jitter; too long and you hold resources past the point of usefulness. A common approach is to derive timeouts from the 99th percentile latency measured in production and multiply by a safety factor of 1.5–2×.

Idempotency is what makes retries safe to use on mutating operations. A POST endpoint that creates an order is not idempotent by default — retrying after a network error could create the order twice. The canonical solution is a client-generated idempotency key (a UUID sent in a request header) that the server stores and uses to deduplicate. Stripe's API has made this pattern famous. For event-driven systems, at-least-once delivery combined with idempotent consumers achieves the same goal.

Retry budgets and deadlines deserve attention at the system level, not just per-service. If Service A retries Service B which retries Service C, a single user request can generate O(n³) backend calls during an outage. Setting a total deadline on the entire request (propagated via a deadline header or gRPC deadline) caps total retry depth and prevents amplification across the call graph.
