How webhooks work
KeyStone dispatches webhooks to all platforms involved in a settlement. Every state transition emits an event, namedsettlement.state. plus the lowercased state name, whatever caused the transition: an engine run, an on-chain outcome, a compliance decision or an operator action. The webhook is written in the same database transaction as the state change, so a transition cannot happen without its announcement.
The single exception is the initial INSTRUCTED state, which is written when the settlement row is created rather than by a transition, and emits nothing. See the event catalog for every event and its payload.
Local development
During development, your server runs onlocalhost which KeyStone can’t reach. The KeyStone CLI solves this by forwarding webhooks to your local server:
Setting up a webhook endpoint
1. Register the endpoint
Create a webhook endpoint in the KeyStone Dashboard under Settings > Webhooks. You need to provide:- URL - Your HTTPS endpoint that will receive webhook deliveries
- Events - Which events to subscribe to, supplied as an
eventsarray of glob patterns (see patterns below). If you omit it, the field defaults to["*"](every event).
secret on the create call instead of letting KeyStone mint one. It must be 32 to 255 characters and, like the URL, may not contain a null character. KeyStone uses the value verbatim as the HMAC-SHA256 key behind X-Keystone-Signature; RFC 2104 recommends an HMAC key no shorter than the hash output, which is 32 bytes for SHA-256, and length is the one property of a key KeyStone can check. A value outside the rule is refused with 422 naming secret, and no endpoint is registered. The refusal never repeats the value you sent. Omit the field and KeyStone mints a 256-bit secret and returns it once.
A description, on create or update, may be at most 500 characters and may not contain a null character; past either it is refused with 422 naming description.
An environment holds at most 50 webhook endpoints. Registering past that returns 422 VALIDATION_ERROR naming the ceiling; delete an endpoint you no longer use before adding another.
Two rules apply to the address your URL reaches, checked at every delivery rather than at registration:
- It must resolve to a publicly routable address. Before each attempt, KeyStone resolves your hostname and connects only if every address in the answer is globally routable; a name that resolves to a private, loopback, link-local, carrier-grade NAT, multicast or otherwise non-routable address is refused without a connection, and the attempt is recorded with
error: "blocked_address". Registration checks less than delivery does: a URL whose host is a private, loopback or link-local address written as a literal is refused at registration with422, while a literal in any other refused class, carrier-grade NAT or multicast for instance, registers and is refused at the first delivery attempt. A hostname is accepted at registration whatever it resolves to, because the answer can change, so the check that counts is the one made before each connection. - Redirects are not followed. A
3xxresponse is a failed attempt recorded witherror: "redirect". Register the final URL.
- A URL may be at most 2048 characters, and at most 2048 bytes once UTF-8 encoded. The two ceilings differ only when the URL contains multibyte characters. A URL past either is refused with
422and adetailarray naming the limit. This is request-body validation, so the response carries the validation shape rather than thecodefield used by the platform’s own error envelope. - A URL may not contain a null character. One that does is refused with
422in that same validation shape. - A URL already registered in the environment is refused with
409 DUPLICATE_RESOURCE.
409 nor any 422 repeats the URL you sent: an endpoint URL is often itself a credential, and response bodies travel into logs and error trackers. The detail array names the field and the rule without carrying the value.
When updating an endpoint, omit a field to leave it unchanged. An explicit null is refused with 422 null_not_allowed for url, events and is_active; only description can be cleared with null.
You can also register endpoints via the API:
2. Event patterns
Patterns are matched with glob semantics (
fnmatch). A pattern that matches no real event is accepted and simply never fires - unknown or misspelled event names are silently ignored, not rejected. If you omit events, it defaults to ["*"].
An endpoint accepts at most 50 patterns, each at most 100 characters and, like the URL, with no null character. Every pattern is matched against every event for every endpoint in the environment, so the list is delivery-path work rather than inert configuration. Both ceilings sit well above the event namespace below. A pattern outside the rule is refused with 422 naming its position in events, on create and on PATCH, and nothing is written.
Webhook events
Event names aresettlement.state. followed by the lowercased state name. Nine of the ten settlement states are delivered, plus the test.ping event; INSTRUCTED is set at creation rather than by a transition and has no event.
There are no separate
settlement.compliance.* events; compliance outcomes surface through the state events above.
Webhook payload format
Every webhook delivery includes:GET /v1/settlements/{settlement_id} when your handler needs more detail.
Verifying signatures
Every webhook delivery includes anX-Keystone-Signature header with a timestamped HMAC-SHA256 signature:
tis the delivery’s Unix timestamp in seconds.v1isHMAC_SHA256(secret, "{t}." + body)in lowercase hex, wherebodyis the raw request bytes.
v1 from the received t and raw body, compare in constant time, and reject deliveries whose t is older than your tolerance window (we recommend 5 minutes) to shut out replays.
Secret rotation
Rotate your webhook secret in the KeyStone Dashboard under Settings > Webhooks by clicking Rotate Secret on the endpoint, or via the API:- Deliveries include both
X-Keystone-Signature(new secret) andX-Keystone-Signature-Previous(old secret) - Your server should verify against both headers
- After 24 hours, only the new secret is used
Testing
Send a test ping from the KeyStone Dashboard under Settings > Webhooks, or via the API:200 whether or not your endpoint answered:
status_code is what your endpoint returned, if it returned anything, and duration_ms the time the attempt took. error is null on success and otherwise one of a fixed set of reason codes, the same set that appears on delivery log entries:
The code names the failure class. Neither surface carries the text of the failure: a certificate error’s chain, a socket error’s message or anything your endpoint wrote in its response stays on KeyStone’s side.
Best practices
- Return 200 quickly - Process webhooks asynchronously. Return a 200 status immediately and handle the event in a background job.
- Handle duplicates - Webhooks may be delivered more than once. Use the settlement ID plus state to make your handler idempotent.
- Verify signatures - Always verify the HMAC signature before processing. Reject unsigned or invalid requests.
- Monitor delivery logs - Check delivery logs in the KeyStone Dashboard or via the API:
Delivery guarantees
Deliveries are at-least-once. Every state transition enqueues its deliveries in the same database transaction that records the transition, and the delivery worker is triggered immediately after that transaction commits - your endpoint receives the webhook within moments of the state change. A scheduled sweep (about once a minute) backstops the immediate trigger, and also picks up a delivery whose trigger arrived while the worker was already mid-sweep, so a committed transition can never lose its webhook and a rolled-back one never sends.- Retries: a failed attempt (anything but a
2xx, including a3xx, or a timeout) is retried on a growing backoff ladder - 1 minute, 5 minutes, 30 minutes, 2 hours, 6 hours, 12 hours, 24 hours - and marked dead after 8 total attempts (about two days). Every attempt appears in the delivery logs with its reason code (the set is listed under Testing). One failure is not retried: a hostname resolving to an address KeyStone does not deliver to (blocked_address) marks the delivery dead on its first attempt, since the record is yours to fix; redeliver or replay it once it resolves publicly. A hostname that does not resolve at all (dns_failure) stays on the ladder. - Timeouts: each request gives your endpoint 10 seconds to answer, and one attempt as a whole is bounded at 60 seconds. Within an attempt, a
429or a connection failure is re-sent a few times before the attempt fails onto the ladder; aRetry-Afteron a429is honoured within the attempt for up to 5 seconds. Ask for longer and the attempt fails instead, and the next one comes on the ladder, whose first rung is a minute. - Deduplication: each delivery carries a stable
X-Keystone-Delivery-Id. Retries and replays reuse the same id - deduplicate on it, and treat your handler as idempotent. A delivery worker that stops after your endpoint acknowledged but before the outcome was recorded re-sends the delivery a few minutes later, so an id you have already acknowledged can arrive again. - Ordering is not guaranteed across events. The payload identifies the settlement’s new state; when ordering matters, fetch the settlement’s current state instead of reconstructing it from event order.
- Your response: only the status code decides the outcome. A
2xxis a delivered webhook; a3xxis a failed attempt (redirects are not followed); anything else is a failed attempt with the status recorded. The body is not read and not stored, and the reason phrase on your status line is not stored either, so acknowledge with an empty200.
Replaying deliveries
Recover missed events after receiver downtime without polling:X-Keystone-Delivery-Id and are signed freshly at send time.
Two bounds apply to the range replay, because every delivery it resets becomes due immediately on a delivery worker shared with every other platform:
- The range may span at most 7 days. A wider range (or one where
to_atprecedesfrom_at) is refused with422and adetailarray naming the ceiling. This is request-body validation, so the response carries the validation shape rather than thecodefield used by the platform’s own error envelope. Branch on the status, not oncode. - Timestamps sent without an offset are read as UTC.
- One call resets at most 200 deliveries, oldest first.
reset_countin the response tells you how many were reset; repeat the call until it comes back below 200 to drain a larger backlog. Already-reset deliveries drop out of the range, so no cursor is needed.
Webhook Event Catalog
See every event type with exact payloads and handler examples.
CLI Webhook Forwarding
Forward webhooks to localhost during development.