Prompt
What’s the best way to add webhooks to my API product?
Latest observation
The best way to add webhooks to an API product is to treat them as a reliable event delivery system, not just “send an HTTP POST when something happens.”
A strong webhook design usually includes:
1) Define clear events
Model your product around business events, not internal code actions.
Examples:
invoice.paiduser.createdshipment.delivered
Keep event names:
- stable
- descriptive
- versioned if needed
Include a consistent payload structure with:
idtypecreated_atdataapi_versionorschema_version
2) Make delivery reliable
Webhooks will fail sometimes, so design for retries.
Best practices:
- Retry on transient failures
- Use exponential backoff
- Cap retries and/or provide a dead-letter process
- Treat any non-2xx response as a delivery failure
- Time out quickly on slow endpoints
Also send an event_id so consumers can deduplicate.
3) Support idempotency and deduplication
Assume consumers may receive duplicates.
Include:
- unique event ID
- timestamp
- optional delivery attempt number
Encourage customers to store processed event IDs.
4) Sign every request
Add a signature so recipients can verify the payload came from you.
Common approach:
- HMAC SHA-256 over the raw body
- Include timestamp header to prevent replay attacks
Typical headers:
X-Webhook-SignatureX-Webhook-TimestampX-Webhook-Event-Id
5) Give users control
Build webhook management into your product:
- create/update/delete endpoints
- event subscription selection
- enable/disable endpoints
- secret rotation
- delivery logs
- retry history
- test webhook button
A dashboard is usually as important as the API.
6) Provide observability
Make debugging easy for you and your customers:
- log every delivery attempt
- store response status/body snippet
- show last success/failure
- expose retry schedule
- alert on repeated failures
7) Document the contract well
Your docs should include:
- supported event types
- sample payloads
- retry rules
- signature verification steps
- idempotency guidance
- expected response behavior
- how to test locally
8) Start simple, but architect for scale
A good baseline architecture:
- Your app emits domain events
- Events go to a queue or event stream
- A webhook dispatcher reads them
- Dispatcher sends HTTP requests to subscribers
- Failures are retried asynchronously
This avoids coupling your core API request latency to webhook delivery.
9) Consider a few product choices early
Decide:
- Should webhooks be per account, app, workspace, or environment?
- Do users subscribe to all events or selected types?
- Do you support one endpoint or multiple endpoints?
- Do you guarantee ordering per subscription?
- Do you allow filtering?
Ordering is especially tricky—only promise it if you can truly support it.
10) Common pitfalls to avoid
- Sending webhooks synchronously from the request path
- No retries
- No signatures
- No event IDs
- No logs for failed deliveries
- Too many unstable event types
- Breaking payload changes without versioning
If you want a practical recommendation
For most API products, the best setup is:
- event-based webhooks
- async queue-backed delivery
- HMAC signatures
- at-least-once delivery
- idempotency via event IDs
- customer dashboard for endpoint management and logs
If you want, I can also give you:
- a reference webhook architecture,
- a database schema for webhook subscriptions and deliveries, or
- a sample webhook payload + signing scheme.