Questions › Networking/Security
Design the client/server contract for a flaky network: rate limiting (429), retries with backoff, and safe retries of non-idempotent requests.
Reasoning about resilient request handling end-to-end — backoff, jitter, Retry-After, and idempotency keys — not just 'add a retry'.
On a real network, requests fail transiently and servers protect themselves, so a robust client and server share a contract. RATE LIMITING: a server caps how many requests a client may make in a window and returns 429 Too Many Requests when exceeded, ideally with a Retry-After header (seconds, or a date) telling the client exactly how long to wait. A well-behaved client HONORS Retry-After rather than hammering. (Servers commonly implement limits with token-bucket or sliding-window algorithms; the client's job is to react correctly to the 429.) RETRIES must be disciplined, or they amplify outages. Rules: (1) Only retry on TRANSIENT failures — network errors, timeouts, 429, and 5xx like 502/503/504; do NOT retry 4xx like 400/401/403/404 (retrying a bad request just repeats a guaranteed failure). (2) Use EXPONENTIAL BACKOFF — wait ~base * 2^attempt (e.g. 0.5s, 1s, 2s, 4s) with a cap — so you back off fast under load instead of retrying instantly. (3) Add JITTER (randomize the delay) — this is critical at scale: without it, many clients that failed at the same instant retry in lockstep and create a synchronized 'thundering herd' that re-crushes the recovering server; randomized delays spread the load. (4) Cap the number of attempts and set an overall deadline so you fail fast rather than retrying forever. The hard part is retrying NON-IDEMPOTENT requests safely. GET/PUT/DELETE are idempotent — a duplicate is harmless — so they're naturally retry-safe. But a POST like 'charge the card' or 'place the order' is dangerous to retry: if the original actually succeeded but the response was lost to a timeout, a blind retry double-charges. The solution is an IDEMPOTENCY KEY: the client generates a unique key (a UUID) for the logical operation and sends it (e.g. Idempotency-Key header) on the request AND on every retry of that same operation. The server records the key with the result of the first successful execution; if it sees the key again, it returns the STORED result instead of executing again. Now a retry is safe — the operation happens at most once regardless of how many times the client resends. (This is exactly how payment APIs like Stripe make POSTs retry-safe.) Putting it together: client uses backoff+jitter, honors Retry-After, retries only transient failures, bounds attempts, and attaches an idempotency key to any non-idempotent write; server enforces limits with clear 429+Retry-After and deduplicates by idempotency key. Bonus resilience: a CIRCUIT BREAKER on the client stops sending after a run of failures (fail fast, give the server room to recover) and probes periodically before closing again.
Building a resilient API client; making a POST retry-safe; handling 429s and outages gracefully.
// Disciplined retry: transient-only, exponential backoff + JITTER, honor Retry-After.
async function request(url, opts = {}, { attempts = 4, base = 500 } = {}) {
for (let i = 0; i < attempts; i++) {
const res = await fetch(url, opts);
if (res.ok) return res;
const transient = res.status === 429 || (res.status >= 500 && res.status <= 599);
if (!transient || i === attempts - 1) return res; // don't retry 4xx; stop at cap
const retryAfter = Number(res.headers.get('Retry-After'));
const backoff = base * 2 ** i; // 0.5s,1s,2s,4s...
const jitter = Math.random() * backoff; // spread the herd
const waitMs = retryAfter ? retryAfter * 1000 : backoff / 2 + jitter;
await new Promise(r => setTimeout(r, waitMs));
}
}
// Non-idempotent POST made retry-safe with an idempotency key:
const key = crypto.randomUUID(); // one key per logical op
await request('/charge', {
method: 'POST',
headers: { 'Idempotency-Key': key, 'Content-Type': 'application/json' },
body: JSON.stringify({ amount: 5000 }),
});
// Server stores result under key; a repeated key returns the SAME result,
// so a timed-out-but-succeeded charge is never double-applied.