I’m building an integration platform that needs to receive webhooks from Stripe and GitHub, what architecture should I use?

For a Stripe + GitHub webhook integration platform, I’d recommend a durable event-ingestion architecture with a thin public webhook edge, message queue, and async workers. The main goals are:

• verify signatures quickly
• return 2xx fast
• avoid losing events on spikes/retries
• process idempotently
• keep provider-specific logic isolated

Recommended architecture

1) Webhook ingress layer

Expose separate endpoints, for example:

• POST /webhooks/stripe
• POST /webhooks/github

This layer should do only a few things:

• read and preserve the raw request body
• verify the provider signature
• normalize metadata into a common envelope
• enqueue the event
• respond 200 OK quickly

Do not do heavy business logic here.

---

2) Verification and normalization

Each provider has different security and payload conventions.

Stripe

• verify using the Stripe-Signature header
• use the raw request body
• validate timestamp tolerance
• extract:
  • event id
  • event type
  • created time
  • object data
  • account / livemode / request info

GitHub

• verify with HMAC SHA-256 using the webhook secret
• support both:
  • X-Hub-Signature-256
  • optionally legacy X-Hub-Signature
• also inspect:
  • X-GitHub-Event
  • X-GitHub-Delivery for uniqueness/idempotency

Normalize both into a common internal format, something like:

{
  "provider": "stripe",
  "provider_event_id": "evt_123",
  "event_type": "invoice.paid",
  "received_at": "...",
  "payload": { ... },
  "headers": { ... },
  "tenant_id": "..."
}

---

3) Durable queue / event log

After verification, publish the event to a durable system:

Good options:

• AWS SQS
• RabbitMQ
• Kafka
• Google Pub/Sub
• Azure Service Bus

If you expect high volume and replay/audit requirements, Kafka or an event log is great.
If you want simpler operational overhead, SQS/Pub/Sub is often enough.

The key requirement: accept first, process later.

---

4) Worker/service layer

A separate worker processes queued events:

• deduplicate
• route by provider/event type/tenant
• transform to internal domain events
• invoke downstream integrations
• store processing status
• retry with backoff
• move poison events to a dead-letter queue

Use a separate worker pool for:

• Stripe
• GitHub
• or even per event family if volume differs

---

5) Idempotency and deduplication

This is critical because both Stripe and GitHub retry deliveries.

Store a processed-event record keyed by:

• Stripe: event.id
• GitHub: X-GitHub-Delivery

Before processing, check whether the event has already been handled.

A typical pattern:

• received_events table with unique constraint on (provider, provider_event_id)
• processing_attempts table for retries and observability
• transactionally mark as processed before side effects if possible

This prevents duplicate downstream actions.

---

6) Internal routing / rules engine

Once normalized, route events to the right handlers:

• Stripe invoice.paid → billing/accounting workflow
• Stripe payment_intent.succeeded → fulfillment
• GitHub push → CI/CD or code indexing
• GitHub pull_request → automation/workflow engine

You can implement this as:

• code-based handlers
• a rules engine
• or a workflow orchestrator like Temporal / Step Functions / Celery workflows

If your platform is meant to integrate many systems, a workflow engine can be very helpful for long-running, retryable tasks.

---

7) Persistence and audit trail

Keep an immutable audit trail of:

• raw webhook body
• parsed envelope
• verification result
• processing status
• retries and errors
• downstream actions taken

This helps debugging and replay.

---

Suggested reference architecture

Stripe/GitHub → API Gateway / Load Balancer → Webhook Ingress Service → Queue/Event Bus → Worker Services → Internal DB / downstream systems

Optionally add:

• Redis for short-lived dedupe or rate limiting
• Dead-letter queue for failures
• Observability stack: structured logs, metrics, traces, alerting
• Secrets manager for provider secrets

---

Important implementation details

Preserve raw body

Both Stripe and GitHub signature verification depends on the exact raw payload.
Avoid JSON parsing before verification.

Fast ACK

Return 200 or 202 within a few hundred milliseconds if possible.

Retry-safe processing

Assume:

• duplicate deliveries
• out-of-order events
• missing events
• temporary downstream failures

Multi-tenant secret management

If you support multiple customer accounts:

• store webhook secrets per tenant/provider
• identify the tenant from route, host, or lookup metadata
• verify against the correct secret

Security

• enforce HTTPS
• verify signatures on every request
• optionally IP allowlist if feasible, but don’t rely on it alone
• log carefully to avoid leaking secrets or PII

---

If you’re starting small

A pragmatic version is:

• Webhook API
• Postgres
• SQS
• Worker service

Use Postgres for:

• dedupe
• event state
• audit trail

Use SQS for:

• buffering
• retries
• decoupling ingress from processing

That’s a strong MVP architecture.

---

When to choose Kafka or an event bus

Use Kafka if:

• high throughput
• need replayability
• many consumers
• event-stream architecture

Use SQS/PubSub if:

• you want simplicity
• moderate volume
• single/few consumer patterns

---

My recommendation

For most integration platforms, start with:

• Webhook ingress service
• Postgres for persistence + idempotency
• SQS/PubSub for async processing
• worker service with retries + DLQ
• structured audit logging

That gives you reliability without overengineering.

If you want, I can also sketch:

1. a concrete AWS architecture,
2. a database schema for webhook events, or
3. example request flow and pseudo-code for Stripe/GitHub verification.