How to Rotate a Payment Webhook Secret Without Downtime

Webhook secret rotation reduces the lifetime of a credential and is mandatory after suspected exposure. It can also interrupt payment confirmation when the sender and merchant endpoint change at different times. A safe rotation treats the change as a small production migration with an owner, a bounded overlap, metrics, and a final removal step.

This guide assumes the endpoint verifies timestamped raw-body HMAC as described in the signed payment webhook guide. Rotation does not change the signed message format or justify skipping replay and idempotency controls.

Prepare the Rotation

Before changing either side, record:

• Rotation owner and approver
• Environment and webhook endpoint
• Current non-secret key identifier
• Application instances that verify events
• Secret distribution mechanism
• Verification-failure alert and dashboard
• Rollback decision and maximum overlap window
• Test event and expected order behavior

Generate the new secret through the authorized product flow. Store it in the same protected secret manager used by the application; do not pass it through email, chat, source control, build logs, or screenshots. Restrict read access to the runtime and the smallest operational group.

Deploy Bounded Dual Verification

When an atomic switch is impossible, deploy code that tries the active new secret and then the retiring secret. Do not accept an event unless one candidate produces a valid timestamped HMAC. Record only a safe key label such as webhook-2026-08 in protected logs.

The overlap implementation should include:

1. Exact raw request bytes captured once.
2. Timestamp freshness checked before business processing.
3. Constant-time digest comparison for each candidate.
4. A metric for which key label verified the event.
5. An explicit expiry for accepting the retiring key.
6. Normal event-ID deduplication after authentication.

Do not leave an unlimited list of historical secrets in the application. Every additional accepted secret expands the credential set an attacker could use.

Activate and Observe

After every application instance can verify both candidates, activate the new secret at the sender. Send a real test event from the Webhook Workbench and confirm:

Evidence	Expected result
Delivery record	Actual 2xx from the intended endpoint
Merchant application log	New key label and matching event ID
Database receipt	One durable event record
Order behavior	No false or repeated transition
Fleet metrics	All active versions use the new key

Watch signature rejection rate, response codes, and instance versions through at least one normal deployment cycle. A single successful request can hit only one instance while another still lacks the new secret.

Retire the Old Secret

Remove the retiring secret from verification code and runtime configuration after the overlap window and fleet evidence are complete. Redeploy, send another test, and confirm no instance requests the old secret. Then remove obsolete secret versions according to your secret-store retention policy.

Close the rotation record with activation time, old-secret removal time, test event IDs, affected versions, anomalies, and approver. Never claim completion while the old value remains accepted by a forgotten worker or standby environment.

Emergency Rotation

Suspected exposure changes the priority. Immediately restrict access, generate a replacement, activate it, and invalidate the exposed value as quickly as the platform allows. A long overlap may be inappropriate because it keeps the compromised secret valid.

At the same time, preserve non-secret evidence: who accessed the secret store, deployment history, signature failures, unusual event IDs, and unauthorized order transitions. Review every event accepted during the exposure window through tenant-scoped records. If payment state may be unreliable, pause affected fulfilment paths and use the payment incident response runbook once available.

Rotation succeeds when event delivery remains authentic and observable—not merely when a value changed in a dashboard.