How do I accept inbound webhooks and enqueue jobs from them?

A common pattern is:

1. Expose a webhook endpoint that receives the HTTP request.
2. Validate/authenticate the request quickly.
3. Parse and normalize the payload.
4. Enqueue a background job with the minimal data needed.
5. Return 2xx immediately so the sender doesn’t retry unnecessarily.
6. Process the job asynchronously in a worker.

Basic flow

Webhook provider -> POST /webhooks/provider
                    -> verify signature
                    -> store/enqueue job
                    -> respond 200 OK
                    -> worker processes job later

Example (Node.js + Express + BullMQ)

1) Webhook route

import express from 'express';
import Redis from 'ioredis';
import { Queue } from 'bullmq';

const app = express();
const redis = new Redis(process.env.REDIS_URL);
const queue = new Queue('webhooks', { connection: redis });

// Important: use raw body if you need to verify signatures
app.post('/webhooks/provider', express.raw({ type: '*/*' }), async (req, res) => {
  try {
    const rawBody = req.body.toString('utf8');

    // 1. Verify signature (pseudo-code)
    // const signature = req.headers['x-signature'];
    // if (!verifySignature(rawBody, signature, process.env.WEBHOOK_SECRET)) {
    //   return res.status(401).send('Invalid signature');
    // }

    // 2. Parse payload
    const payload = JSON.parse(rawBody);

    // 3. Enqueue job
    await queue.add('process-webhook', {
      provider: 'provider',
      payload,
      receivedAt: new Date().toISOString(),
      eventId: payload.id ?? null,
    }, {
      attempts: 5,
      backoff: {
        type: 'exponential',
        delay: 1000,
      },
      removeOnComplete: true,
      removeOnFail: 1000,
    });

    // 4. Acknowledge immediately
    res.status(200).send('OK');
  } catch (err) {
    console.error(err);
    res.status(400).send('Bad Request');
  }
});

2) Worker

import { Worker } from 'bullmq';
import Redis from 'ioredis';

const worker = new Worker('webhooks', async job => {
  const { payload } = job.data;

  // Do the real work here
  console.log('Processing webhook:', payload);

  // Example: update DB, call APIs, send emails, etc.
}, {
  connection: new Redis(process.env.REDIS_URL),
});

Important best practices

1) Verify authenticity

Most webhook providers sign requests. Verify:

• HMAC signature header
• timestamp to prevent replay attacks
• shared secret or public key

2) Make processing idempotent

Webhooks are often delivered more than once. Store an event ID and ignore duplicates.

Example:

// before enqueueing or before processing
// if (await alreadyProcessed(eventId)) return res.status(200).send('OK');

3) Respond fast

Webhook senders usually expect a quick 2xx. Don’t do heavy work in the HTTP request.

4) Persist enough context

If payloads are large or can change, consider:

• storing the raw payload in object storage/db
• enqueueing only IDs and metadata

5) Retry safely

Use:

• job retries with exponential backoff
• dead-letter handling / failure queue
• alerting on repeated failures

6) Protect the endpoint

• HTTPS only
• rate limiting
• IP allowlists if supported
• size limits on request body
• CSRF not applicable for server-to-server webhooks

If you want a minimal architecture

• Webhook endpoint: receives and validates request
• Redis / queue: stores jobs
• Worker process: consumes jobs
• Database: tracks processed event IDs and job status

Pseudocode pattern

on POST /webhook:
  verify signature
  event = parse body
  if event.id already processed:
    return 200
  enqueue event
  return 200

worker:
  dequeue event
  process event
  mark event processed

If you want, I can show the same pattern in Python (FastAPI/Celery), Go, or Rails.