How do retries, dead-letter queues, and failure destinations work?

Begin with the invocation model

For synchronous calls, the caller receives the response and normally owns retry policy. For asynchronous Lambda invocation, Lambda queues the event and applies configured retry and age behavior. For queues and streams, the event source and mapping control visibility, checkpoints, batches, and redelivery.

Do not apply one generic retry rule to every path. Record which component owns the attempt count, backoff, timeout, maximum age, and terminal destination.

Retry only failures that can plausibly recover

Timeouts, throttling, temporary dependency unavailability, and capacity pressure may recover after delay. Invalid input, failed authorization, unsupported schema, and permanent business rejection usually require correction rather than immediate repetition.

Use bounded exponential backoff with jitter where you control the client. Align total retry duration with the business deadline and reserve enough capacity for recovery traffic after the dependency returns.

Isolate poison work without losing context

A DLQ or failure destination should preserve the original event reference, failure reason, attempts, timestamps, source, function version, and correlation identifiers without leaking secrets. Configure alarms before the first message arrives.

Isolation protects healthy traffic from a permanently failing item. It does not decide whether to discard, repair, compensate, or replay that item, so ownership and retention must be explicit.

Build a controlled redrive process

Investigate whether failure came from code, configuration, permission, dependency, or data. Correct the cause, validate a sample, and replay at a rate the recovered system can absorb.

Redrive must preserve idempotency and ordering rules. Record who initiated it, which items were selected, the deployed version, outcomes, and any items still requiring manual resolution.