Your client sends “create payment,” the connection times out, and no one knows whether the server charged the customer. Retrying without an operation identity can create a second charge; refusing to retry can strand an order. An idempotency key gives the server a durable way to recognize a retry of the same intended operation and return its recorded outcome.
Idempotency is not “ignore duplicate HTTP requests.” It is a contract: for a defined scope and retention period, the same operation key with the same input maps to one logical effect and a repeatable result. It does not guarantee exactly-once network delivery or make unrelated side effects atomic.
Generate a high-entropy key once when the user or job creates an intent, then persist it before sending. A random UUID is often adequate. Do not use a timestamp alone, a customer ID, or the current retry number. Scope the key by tenant/account and operation so unrelated requests do not collide. Never let an untrusted client choose a key that can cross another tenant boundary.
Store a canonical request fingerprint with the key. Reusing a key with different amount, currency, destination or body should return a conflict, not silently replay the old result or execute a new mutation. Normalize values consistently before hashing; avoid including incidental transport fields that do not alter business intent.
| State | Meaning | Retry behavior |
|---|---|---|
| New | No operation record exists for scoped key. | Atomically claim before executing. |
| In progress | One request owns execution; another attempt arrived concurrently. | Return a documented conflict/retry-after or wait briefly. |
| Completed | Effect and stable response are recorded. | Return the saved outcome for matching fingerprint. |
| Failed / unknown | Outcome may be definitive or an external side effect is uncertain. | Classify carefully; reconcile before creating another effect. |
Use a unique database constraint on (scope, idempotency_key) to arbitrate simultaneous requests. Persist status, request fingerprint, response code/body or a stable result reference, and timestamps. For a mutation in the same database, write the business row and completed idempotency result in one transaction. If execution calls an external payment or email API, use that provider's idempotency mechanism too, or use an outbox/reconciliation workflow: your local transaction cannot atomically commit a remote HTTP side effect.
An order service creates payment intent key `order-842:authorize-v1` and submits it. The payment provider authorizes, but the response is lost. The service retries with the same provider key and receives the original authorization result. It then stores the provider reference and marks its own operation complete. A new authorization attempt after a deliberate decline or changed amount is a new business intent with an explicit new key and audit trail.
Provider semantics differ. Some cache the first response, including certain server errors; some retain keys only for a documented period; some reject parameter mismatches. Read the API contract and set your local retention longer than the maximum retry/reconciliation window. After a provider's key expires, do not blindly retry an uncertain charge; query/reconcile first.
Persist the provider delivery ID with a unique constraint to deduplicate transport redelivery. Then apply domain idempotency too: update a subscription only if the incoming source version is newer or the state transition is still valid. A provider may send two different event IDs about the same object, so delivery dedupe alone is not enough. Keep delivery attempts and business effects as separate records.
Queue messages may be delivered again after a worker crash. Give the logical job a stable operation ID, claim it atomically and make each downstream action idempotent. For multi-step workflows, record step state and use an outbox/saga with compensating actions where appropriate. “Exactly once” is usually an illusion across independent systems; aim for at-least-once delivery with one logical effect and reconciliation.
Deleting keys too early allows a late retry to execute again; retaining them forever increases storage and privacy obligations. Define retention from maximum client retry, queue redelivery, offline device and support-replay windows. Preserve a compact tombstone or business uniqueness record after dropping response bodies where duplicate effects would be costly. Expiration is part of the API contract.
Test same key/same body sequentially and concurrently, same key/different body, crash before commit, crash after commit before response, provider timeout after remote success, key expiration and replay after recovery. Assert both returned response and number of business effects. Log a correlation ID and key hash, not sensitive raw payloads or payment credentials.
An idempotency key names one intended mutation across unreliable delivery. Generate it once, scope it correctly, bind it to a request fingerprint, claim it under a uniqueness constraint, persist outcomes coherently and understand external provider retention. Retries then become controlled recovery rather than a gamble that duplicates money or work.