Handling webhooks in production

Webhooks are how Octopus Cards tells the integration when something terminal happens — an async voucher order completed, a top-up failed, an eSIM was issued. A single endpoint serves all three verticals with the same signing scheme, headers, and dispatch semantics; only the event type and the data payload differ.

A robust webhook consumer has four jobs: verify authenticity, deduplicate, ack fast, process reliably. This guide walks through all four with examples in Go, Node.js, and Python.

For the low-level signing mechanics, see Signature Verification. For event-by-event payload shapes, see Voucher Webhooks, Topup Webhooks, and eSIM Webhooks.

The four jobs

1. Verify — the request really came from Octopus Cards (HMAC-SHA256 over the raw body).
2. Deduplicate — on X-Event-ID. Octopus Cards retries on non-2xx, and may re-deliver after recoveries.
3. Ack — 2xx within a few seconds.
4. Process — asynchronously on the integration side.

A common mistake is collapsing steps 3 and 4: doing all the business logic inline before responding 200 OK. That leads to timeouts and retry storms. Ack first, process second.

Endpoint requirements

• Public HTTPS URL. HTTP is rejected; self-signed certs are rejected.
• Accepts POST with Content-Type: application/json.
• Responds within a few seconds with any 2xx. Non-2xx triggers retry.
• Response body is ignored. Return empty 200 OK or a small JSON ack — either works.

Headers we send

The event envelope

Every event uses the same envelope; the type field discriminates and the data payload is vertical-specific:

{
  "id": "evt_01HYZABC12DEF34GHI56JK",
  "type": "order.delivered",
  "created_at": "2026-04-22T17:30:00Z",
  "data": { /* event-specific payload */ }
}

Live event types

Until topup and eSIM webhooks ship, use the polling pattern from Sync vs async orders — same poll loop, different URL per vertical. Voucher webhooks are live today.

End-to-end handler

The same handler shape works for all three verticals — verify, dedup, ack, hand off. The examples below use Redis for deduplication, but any atomic set-if-not-exists store works (Postgres unique index, DynamoDB conditional put, etc.).

package webhooks

import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/hex"
    "io"
    "net/http"
    "strconv"
    "time"

    "github.com/redis/go-redis/v9"
)

type Handler struct {
    Secret string
    Redis  *redis.Client
    Queue  chan []byte // hand off to your own async worker
}

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // 1. Read the raw body — verification needs the exact bytes
    body, err := io.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "bad body", http.StatusBadRequest)
        return
    }

    // 2. Verify the HMAC signature (constant-time)
    receivedSig := r.Header.Get("X-Signature")
    mac := hmac.New(sha256.New, []byte(h.Secret))
    mac.Write(body)
    expected := hex.EncodeToString(mac.Sum(nil))
    if !hmac.Equal([]byte(receivedSig), []byte(expected)) {
        http.Error(w, "bad signature", http.StatusUnauthorized)
        return
    }

    // 3. Reject stale events (replay protection)
    ts, _ := strconv.ParseInt(r.Header.Get("X-Timestamp"), 10, 64)
    if time.Since(time.Unix(ts, 0)) > 5*time.Minute {
        http.Error(w, "stale", http.StatusBadRequest)
        return
    }

    // 4. Deduplicate on X-Event-ID
    eventID := r.Header.Get("X-Event-ID")
    ctx := r.Context()
    set, err := h.Redis.SetNX(ctx, "webhook:"+eventID, 1, 24*time.Hour).Result()
    if err != nil {
        // Storage hiccup — let Octopus Cards retry
        http.Error(w, "server busy", http.StatusServiceUnavailable)
        return
    }
    if !set {
        // Already processed — ack and move on
        w.WriteHeader(http.StatusOK)
        return
    }

    // 5. Ack first, process second
    select {
    case h.Queue <- body:
        w.WriteHeader(http.StatusOK)
    default:
        // Queue saturated — unset the dedup key so a retry can re-attempt
        _ = h.Redis.Del(ctx, "webhook:"+eventID).Err()
        http.Error(w, "busy", http.StatusServiceUnavailable)
    }
}

type event struct {
    ID        string          `json:"id"`
    Type      string          `json:"type"`
    CreatedAt time.Time       `json:"created_at"`
    Data      json.RawMessage `json:"data"`
}

func worker(queue <-chan []byte) {
    for body := range queue {
        var ev event
        if err := json.Unmarshal(body, &ev); err != nil {
            log.Printf("malformed event: %v", err)
            continue
        }
        switch ev.Type {
        case "order.delivered", "order.partially_delivered":
            handleVoucherDelivered(ev)
        case "order.failed", "order.cancelled":
            handleVoucherTerminal(ev)
        case "topup.delivered", "topup.failed", "topup.cancelled":
            handleTopupTerminal(ev)
        case "esim.delivered", "esim.failed", "esim.cancelled":
            handleEsimOrderTerminal(ev)
        case "esim.installed", "esim.activated", "esim.depleted":
            handleEsimLifecycle(ev)
        default:
            log.Printf("unknown event type %q — ignoring (this is fine)", ev.Type)
        }
    }
}

The worker (drains queue, runs business logic) is where time-consuming work lives — never inline in the HTTP handler.

Voucher-specific: fetch the order to get the codes

order.delivered and order.partially_delivered confirm fulfilment but do not include the actual voucher codes (PINs / card numbers) in the payload — those are sensitive single-use credentials and webhook payloads are logged and retried.

After the webhook, call GET /api/v1/orders/:id over the authenticated channel to retrieve the codes:

func handleVoucherDelivered(ev event) {
    var data struct {
        ID              uint64  `json:"id"`
        ClientReference *string `json:"client_reference,omitempty"`
    }
    _ = json.Unmarshal(ev.Data, &data)

    url := fmt.Sprintf("%s/api/v1/orders/%d", os.Getenv("HOST"), data.ID)
    req, _ := http.NewRequest("GET", url, nil)
    req.Header.Set("Authorization", "Bearer "+os.Getenv("TOKEN"))

    resp, _ := http.DefaultClient.Do(req)
    defer func() { _ = resp.Body.Close() }()

    // Decode order, including voucher codes; persist on your side
    // before the user closes the page — codes will be masked on
    // future fetches.
}

The same "fetch on receipt" pattern will apply when eSIM webhooks ship — the activation_code is omitted from esim.delivered payloads for the same single-use-credential reason.

Deduplication storage

Any store that supports atomic "set-if-not-exists" works:

• Redis SETNX (or SET … NX EX 86400) — simple and fast. The examples above use this.
• Postgres with a unique index on event_id and INSERT … ON CONFLICT DO NOTHING.
• DynamoDB conditional PutItem (ConditionExpression: 'attribute_not_exists(event_id)').

Don't use in-process maps — they reset on deploy and lose deduplication across replicas.

Retry semantics

• Non-2xx responses (and timeouts) are retried at least once.
• Retries are not instantaneous — expect minutes of delay on the second attempt.
• Retries carry the same X-Event-ID. The dedup key catches them.
• After several attempts Octopus Cards gives up. The integration is then in a gap state; reconcile via polling.

Ordering

Webhooks are not strictly ordered. Two order.delivered events for different orders can arrive in any order; a retry can even arrive after a newer event for a different order.

Rules:

• Don't rely on arrival order for correctness.
• Each event payload carries enough state (id, status, etc.) to be processed independently.
• If an integration genuinely needs an ordering key, use created_at from the envelope — not arrival time.

Response-time budget

There is a ~5 second window before the request is considered timed out. Design accordingly:

Total well under 100 ms in the happy path — leaving headroom for GC, network, TLS handshake, etc. If the handler is close to 5s, something is wrong.

Testing

In sandbox

Trigger a webhook by placing an async (quantity > 5) sandbox voucher order. order.delivered fires within seconds.

Locally

Use a tunnelling tool (ngrok, Cloudflare Tunnel) to expose localhost. Point the sandbox webhook URL at the tunnel. This lets the handler logic be debugged without deploying.

Simulated failures

Force the handler to return 500 intermittently and watch the retry come in. This validates:

• The dedup store holds the event ID across attempts.
• The worker doesn't double-process when it partially succeeded before the 500.
• The 2xx response unsets the retry path.

Security — the non-negotiables

• Always verify HMAC. An unverified webhook endpoint is an open invitation for anyone to inject completion events.
• Constant-time comparison. Use hmac.Equal in Go, crypto.timingSafeEqual in Node, hmac.compare_digest in Python. Never plain ==.
• Reject if X-Timestamp is missing or implausible. Replay protection depends on it.
• HTTPS only. TLS termination in front of the handler, always.

Full threat model: Security.

Summary

• Same envelope, same headers, same handler shape across all three verticals.
• Voucher webhooks are live today (order.delivered, order.partially_delivered, order.failed, order.cancelled). Topup and eSIM webhooks are planned — use polling in the meantime.
• Four steps in every handler: verify → dedup → ack → process. Never inline the business logic in the HTTP handler.
• Retries are at-least-once, unordered, and carry the same X-Event-ID. Dedup catches them.
• Sensitive credentials (voucher codes, eSIM activation codes) are not in webhook payloads — fetch the order over the authenticated channel after receiving the event.