Duplicates are normal operating conditions
Approval workflows cross browsers, email clients, APIs, queues, and workers. Every boundary can retry. A person can double-click a decision button, a client can repeat a timed-out create request, and a webhook provider can redeliver an event after the receiver committed its work but lost the response.
Trying to remove every duplicate is unrealistic. Production systems become reliable by making duplicates safe. That starts with an atomic decision, continues through idempotent event delivery, and ends with a side effect that has its own deduplication boundary.
Allow one terminal decision to win
Approve and reject can arrive nearly simultaneously from separate tabs or devices. The decision store must transition the record from pending to one terminal state in a single conditional operation. If the record is no longer pending, the later request returns the existing outcome rather than overwriting it.
This rule belongs in the database transaction, not only in a disabled button. Interface controls improve the experience, but they cannot coordinate concurrent requests or a replayed HTTP call.
UPDATE approvals
SET status = 'approved', decided_at = NOW()
WHERE id = :approval_id
AND status = 'pending';
-- Exactly one row means this request won.
-- Zero rows means return the existing terminal state.Give creation requests a stable identity
A create call can succeed even when the caller never receives the response. Without an idempotency key, retrying may email the same person twice and produce two valid approval records for one workflow action.
Derive a key from the durable operation identity, not from a random value created on each attempt. Store a hash of the normalized request body with the key. Reusing the same key and body returns the original approval; reusing it with a different body should fail as a conflict. That protects callers from accidental parameter drift.
- Choose keys that identify the business operation, such as deploy-prod-1042 or refund-order-9281.
- Keep the same key across network retries and worker restarts.
- Reject the same key with materially different recipient, action, or context fields.
Treat webhook delivery as an inbox
A signed webhook proves where an event came from; it does not prove that your application has not handled it before. Record the stable event ID in an inbox table under a unique constraint. Verify the signature and timestamp against the raw body, then insert the ID and apply the local transition in the same transaction where possible.
A duplicate event should receive a successful response after confirming it was already handled. Returning an error invites more retries without improving correctness. Keep enough event metadata to explain which delivery caused a workflow to resume.
- Verify HMAC signatures with a constant-time comparison.
- Reject events outside the accepted replay window.
- Use the provider event ID, not a hash of mutable JSON formatting, as the deduplication key.
- Queue execution after recording the event so slow side effects do not block delivery acknowledgement.
Protect the last mile too
An atomic approval does not automatically make the resulting deploy, refund, or message exactly once. The worker can fail after calling the downstream system but before recording success. Pass the same business idempotency key to downstream APIs that support one, or store an execution ledger around systems that do not.
Track approval status and execution status separately. Approved means a human authorized the action; it does not mean the action completed. This distinction gives operators truthful recovery options and prevents a retry from asking for a second approval when only execution needs to resume.
Test the races deliberately
Happy-path tests rarely expose duplicate execution. Add concurrency tests that submit approve and reject together, repeat create requests with the same idempotency key, deliver the same webhook many times, and crash a worker after the downstream call but before its local commit.
Exactly-once is not a single feature. It is a chain of explicit identities, atomic transitions, and idempotent consumers. When each boundary has a defined invariant, retries stop being incidents and become a normal recovery mechanism.
Build a checkpoint your application can trust.
Start with one approval request, then connect the signed outcome to your workflow.