Idempotency

Understand This First

• State – idempotency requires tracking whether an operation has already been applied.
• Database – idempotency keys and deduplication records are typically stored in a database.
• Atomic – checking for a duplicate and executing the operation must be atomic to prevent race conditions.
• Transaction – idempotency checks are often implemented within a transaction.

Context

In real systems, operations fail and get retried. A network request times out and the client sends it again. A message queue delivers a message twice. A user double-clicks a submit button. If the operation creates a second order, charges the credit card again, or inserts a duplicate record, the system has a serious problem. Idempotency is the property that running an operation multiple times produces the same result as running it once. This is an architectural pattern because it affects the design of APIs, message handlers, and data operations throughout a system.

Problem

How do you make operations safe to retry without causing unintended side effects?

The internet is unreliable. A client sends a request to create an order. The server processes it successfully, but the response is lost in transit. The client, seeing no response, retries. If the “create order” operation isn’t idempotent, the customer now has two identical orders. The same problem appears with message queues (at-least-once delivery means duplicates), background jobs (a crashed worker may have finished before the crash was detected), and user interfaces (double submissions).

Forces

• Reliability demands retries. You can’t trust that every operation will succeed on the first attempt.
• Naive retries of non-idempotent operations cause duplicates, double charges, and data corruption.
• Making operations idempotent adds complexity to the implementation.
• Not all operations are naturally idempotent; creation and deletion behave differently from updates.

Solution

Design operations so that executing them more than once has the same effect as executing them once.

Some operations are naturally idempotent. Setting a value (“set the user’s email to alice@example.com”) is idempotent because doing it twice produces the same result. Deleting by ID (“delete record #42”) is idempotent because the second delete finds nothing to delete and is a no-op. Reading data is inherently idempotent.

Other operations aren’t naturally idempotent and require explicit design. The most common technique is the idempotency key: the client generates a unique identifier for each logical operation and sends it with the request. The server checks whether it has already processed a request with that key. If it has, it returns the previous result instead of executing the operation again.

POST /orders
Idempotency-Key: abc-123-def-456
{ "item": "widget", "quantity": 1 }

The first time the server sees abc-123-def-456, it creates the order and stores the result keyed by that ID. If the same key arrives again, it returns the stored result without creating a second order.

Other approaches include using database constraints (a unique index prevents duplicate records), using upsert operations (insert-or-update instead of insert), and designing state machines where reprocessing a message that has already been applied is a no-op because the state has already moved past that step.

How It Plays Out

A payment processing system handles credit card charges. A charge request times out and the client retries. Without idempotency, the customer is charged twice. With an idempotency key, the second request is recognized as a duplicate and the original charge result is returned. No double billing, no customer complaint, no refund workflow.

AI agents generating API endpoints almost never implement idempotency unless explicitly asked. An agent asked to “create a POST endpoint for orders” will produce a handler that creates a new order on every call. Adding “make the create-order endpoint idempotent using an idempotency key header” to the prompt produces a handler with duplicate detection built in. This is one of those details that separates prototype-quality code from production-quality code.

Consequences

Idempotent operations make retry logic safe and simple. The client can retry freely without worrying about side effects, which makes the system more resilient to network failures, timeouts, and duplicate message delivery. It simplifies error handling throughout the stack because “when in doubt, retry” becomes a viable strategy.

The costs are implementation complexity and storage. Idempotency keys must be stored and checked, which adds a lookup to every request. The stored results must be retained long enough for retries to arrive (typically minutes to hours), which means additional storage and cleanup logic. Idempotency across distributed systems, where the same logical operation may touch multiple services, requires coordination that isn’t trivial to implement correctly.

Related Patterns

• Uses / Depends on: State — idempotency requires tracking whether an operation has already been applied.
• Uses / Depends on: Database — idempotency keys and deduplication records are typically stored in a database.
• Enables: Consistency — idempotent operations prevent duplicate-induced inconsistencies.
• Uses / Depends on: Atomic — checking for a duplicate and executing the operation must be atomic to prevent race conditions.
• Uses / Depends on: Transaction — idempotency checks are often implemented within a transaction.
• Refines: CRUD — idempotency is a refinement of create and update operations for reliability.