Designing resends, ordering, and idempotency

ReceiptRoller retries deliveries your server did not accept. As a result the same event can arrive more than once, events can arrive out of order, and occasionally one does not arrive at all. This article sets out the retry rules and how to build a receiver that copes with them.

Success and failure

• A 2xx response is a success
• Anything else is a failure and is retried, 4xx (including 401 and 404) is treated the same as 5xx
• No response within 10 seconds, or a DNS, TLS or connection error, is also a failure. The delivery log shows httpStatus 0 for these

Retry timing

• Four attempts at most; the last comes about 71 minutes after the first (plus up to 10 seconds of waiting per attempt)
• A retry has the same body, signature and X-RR-Delivery as the first attempt
• If all four fail, the webhook is switched off (active: false) and no further events are sent to it. Four failures of a single event are enough to stop the whole webhook
• Events that happen while a webhook is switched off are not held back and do not arrive after you switch it back on. There is no way to resend a past event by hand
• Pausing a webhook (active: false) does not stop retries that are already waiting

Retries are not stored

Pending retries are held on a ReceiptRoller server, not saved to a queue or database. If that server restarts while a retry is waiting (during a deployment, for example), the event's remaining retries do not happen and the webhook is not switched off either. The delivery log just ends with the last failed attempt.

So do not assume that retries mean every event arrives. Where a missed event matters, reconcile against the API on a schedule. For sales, the Transactions API returns them for a period you specify.

Order is not guaranteed

Each event is sent on its own and retries arrive later, so for example:

• An order's order.updated arrives before its order.created
• A sale whose first attempt failed arrives after later sales

Where order matters, treat the webhook as a signal that something changed and fetch the current state from the API. The body's timestamp is when ReceiptRoller sent the event out and does not change on retries, so you can compare it to avoid overwriting an order's newer state with an older notification.

Dropping duplicates

An event mostly arrives twice when your server finished processing but did not return 2xx (it took longer than 10 seconds, or failed part-way). Every delivery has an X-RR-Delivery header that is unique per event and equal to the body's id; use it as the key.

async function handle(deliveryId, event) {
  // already processed: just return 2xx
  const inserted = await db.processedDeliveries.insertIfAbsent({
    delivery_id: deliveryId,
    received_at: new Date(),
  });
  if (!inserted) return;

  await processEvent(event);
}

• Keep processed IDs for longer than the roughly 71 minutes retries can take; a few hours at least
• If you can record the ID and update your data in one transaction, a failure part-way cannot leave the ID recorded without the update
• If several of your webhooks subscribe to the same event, deliveries from all of them carry the same X-RR-Delivery, so you process the event once

You can also make writes idempotent with a business key. For a sale, upsert by data.id (in the form {terminal_id}|{yyyy-MM}|{transaction_id}); for an order, by data.order_id. Then the same content arriving twice changes nothing.

A receiver that works well

• Verify the signature
• Check X-RR-Delivery; if it was already processed, return 200 at once
• Save the body to your own queue or table
• Return 200 straight away (well within 10 seconds)
• Do heavy work and calls to other services from the saved data in a separate process, retrying on your side

That way an outage in another service does not make deliveries fail, and you avoid four failures switching the webhook off.

Common mistakes

Related guides

• Webhook overview
• Signature verification and security
• Monitoring and handling failures
• Receive new sales by webhook and post them to Slack (quickstart)