Prompt
How do I troubleshoot a webhook integration that stops working
Latest observation
To troubleshoot a webhook integration that suddenly stops working, go from the outside in: confirm the event is being generated, then trace delivery, then inspect your receiver, then verify retries and auth.
1) Confirm the sender is still generating the event
- Trigger the action that should fire the webhook.
- Check the source system’s event/audit logs, if available.
- Make sure the integration hasn’t been disabled, rotated, or deleted.
- Verify you’re testing the right environment:
- production vs staging
- correct account/org/project
- correct endpoint URL
2) Check delivery attempts in the provider
Look at the webhook provider’s delivery logs or dashboard and inspect:
- HTTP status codes returned
- response body
- timestamps
- retry attempts
- timeout errors
- DNS or connection errors
Common clues:
- 2xx means the provider thinks delivery succeeded.
- 3xx can break some webhook senders if redirects are not followed.
- 4xx usually means your endpoint rejected the request.
- 5xx usually means your server errored or was unavailable.
- timeout / connection refused / DNS error suggests network or host issues.
3) Verify your endpoint is reachable
From an external machine, test:
- DNS resolution
- TLS certificate validity
- firewall rules
- load balancer / reverse proxy config
- whether the endpoint requires VPN/IP allowlisting
Useful checks:
curl -v https://your-webhook-url- confirm port 443 (or your chosen port) is open
- ensure the endpoint is public and not behind internal auth
4) Inspect your server logs
Check logs for the exact time the webhook should have arrived:
- request path
- headers
- payload
- response status
- exceptions/errors
If nothing appears in logs:
- traffic may not be reaching your server
- the URL may be wrong
- a proxy/WAF may be blocking it
If you do see requests:
- verify your app handles the HTTP method the sender uses, usually
POST - ensure it reads the raw body correctly
- confirm it returns a fast response, ideally within a few seconds
5) Validate payload handling
Webhook parsers often break when the payload changes slightly. Check for:
- missing required fields
- schema changes from the provider
- unexpected nested data
- character encoding issues
- gzip/compressed payloads
- empty body or malformed JSON
If you verify signatures, ensure you:
- use the raw request body
- use the correct signing secret
- match the provider’s current algorithm/version
- account for header name changes
6) Check authentication and security layers
Webhooks can stop working because of:
- expired API keys or signing secrets
- rotated secrets not updated on your side
- IP allowlists that changed
- WAF/CDN rules blocking the sender
- basic auth or token changes
- certificate expiration
Also verify:
- your endpoint allows the sender’s user agent/IPs if applicable
- you’re not rejecting requests due to replay protection clock drift
7) Look at retries and idempotency
Many providers retry if they don’t get a successful response. Check whether:
- the provider is retrying and your endpoint is failing consistently
- your app processes the same webhook more than once
- your app is incorrectly marking processed events as failed
Make sure your handler is idempotent:
- use event IDs to deduplicate
- avoid side effects on repeated delivery
8) Reproduce with a manual test
Try sending a known-good request to your endpoint:
- use
curlor Postman with the same method, headers, and body - compare against a real webhook payload
- confirm the same route and middleware are used
Example:
curl -X POST https://your-webhook-url \
-H "Content-Type: application/json" \
-d '{"test":true}'
9) Check timeouts and performance
If your endpoint is slow:
- the sender may time out before your app responds
- reverse proxies may cut the connection
- queued jobs may delay processing too much
Best practice:
- acknowledge quickly with a
200/202 - process the payload asynchronously
10) Verify changes in infrastructure or deployment
Recent changes often cause webhook failures:
- new release or config change
- changed route/path
- TLS/certificate renewal issue
- server restart or autoscaling problem
- proxy/load balancer changes
- database outage causing handler failures
11) Use a webhook replay tool if available
Some providers let you:
- replay a failed event
- resend from the dashboard
- inspect request/response history
That’s one of the fastest ways to isolate whether the issue is delivery or receiver-side logic.
12) If all else fails, create a minimal test endpoint
Temporarily point the webhook to a simple endpoint that:
- logs headers and body
- returns
200immediately
If that works, the problem is in your application logic, middleware, auth, or payload parsing.
Quick checklist
- Event is actually being generated
- Webhook URL is correct
- Provider delivery logs show attempts
- Endpoint is publicly reachable
- Server logs show incoming requests
- Response is 2xx and fast
- Signature/auth is valid
- No firewall/WAF/CDN blocks
- Payload schema hasn’t changed
- Retries/idempotency are handled
If you want, I can also give you:
- a step-by-step debugging flowchart, or
- a checklist specific to your webhook provider (Stripe, GitHub, Shopify, Twilio, etc.).