Prompt
How do I build a PDF generation workflow with retries and webhooks?
Latest observation
A solid PDF-generation workflow usually has 4 pieces:
- API request comes in
- A background job generates the PDF
- Retries handle transient failures
- Webhooks notify downstream systems when done/failed
Here’s a practical way to design it.
1) Basic architecture
Recommended flow
- Client sends a request to your API: “generate PDF for document X”
- Your API validates it and creates a job record
- The job is pushed to a queue
- A worker picks it up and generates the PDF
- Worker stores the PDF in object storage (S3, GCS, etc.)
- Worker updates job status in DB
- Worker sends a webhook event like
pdf.completedorpdf.failed
Why this structure works
- API stays fast
- Workers can retry safely
- You can scale PDF generation independently
- Webhooks let other services react without polling
2) Job state model
Use a simple status machine:
queuedprocessingsucceededfailedretrying
Store at least:
job_idtenant_id/user_id- input parameters
- current status
- retry count
- last error
- output file URL
- timestamps
Example DB row:
{
"job_id": "job_123",
"status": "processing",
"attempt": 2,
"max_attempts": 5,
"output_url": null,
"last_error": "Chrome crashed",
"created_at": "2026-10-12T10:00:00Z"
}
3) Retry strategy
Retries should be used for transient failures, not permanent ones.
Good retry candidates
- browser/renderer timeout
- temporary network issues
- storage upload failure
- dependency unavailable
- rate limits
Don’t retry automatically
- invalid HTML/template
- missing required data
- malformed input
- permission errors
Retry rules
- Use exponential backoff
- Add jitter to avoid retry storms
- Set a max retry count
- Preserve idempotency so duplicate attempts don’t create duplicate outputs
Example backoff:
- attempt 1: immediate
- attempt 2: 2s
- attempt 3: 10s
- attempt 4: 30s
- attempt 5: 2m
4) Webhook design
Send webhook events to consumers whenever the job changes state.
Common events
pdf.job.createdpdf.job.startedpdf.job.completedpdf.job.failedpdf.job.retrying
Webhook payload example
{
"event": "pdf.job.completed",
"job_id": "job_123",
"status": "succeeded",
"output_url": "https://storage.example.com/pdfs/job_123.pdf",
"attempt": 3,
"timestamp": "2026-10-12T10:05:30Z"
}
Best practices for webhooks
- Sign payloads with HMAC
- Include an
event_idfor deduplication - Retry webhook delivery separately from PDF generation
- Use HTTPS only
- Support response codes:
2xx= success5xx/timeouts = retry4xx= usually no retry unless429
5) Separate retries for jobs and webhooks
Treat these as two independent systems:
A. PDF generation retries
Retry when the worker fails to generate or upload the PDF.
B. Webhook delivery retries
Even if the PDF succeeds, webhook delivery may fail. Queue webhook events and retry delivery separately.
This avoids situations where:
- PDF is successfully created
- webhook endpoint is down
- you accidentally regenerate the PDF repeatedly
6) Idempotency
This is important.
For generation requests
If a client retries the API request, don’t create duplicate jobs. Use an idempotency key:
- client sends
Idempotency-Key - same key + same tenant = same job result
For webhook delivery
Use event_id so consumers can deduplicate repeated deliveries.
7) Suggested sequence
Request flow
POST /pdf-jobs- API validates input
- API creates job with
queued - API enqueues job and returns
202 Accepted
Worker flow
- Worker picks job
- Marks
processing - Generates PDF
- Uploads PDF
- Marks
succeeded - Enqueues webhook event
Failure flow
- Worker fails
- Classify error as transient or permanent
- If transient:
- increment attempt
- set
retrying - requeue with backoff
- If permanent:
- mark
failed - send failure webhook
- mark
8) Example API responses
Create job
POST /pdf-jobs
202 Accepted
{
"job_id": "job_123",
"status": "queued"
}
Check job
GET /pdf-jobs/job_123
{
"job_id": "job_123",
"status": "succeeded",
"output_url": "https://..."
}
9) Reliability tips
- Use a durable queue: SQS, RabbitMQ, Redis queue, Kafka, etc.
- Use a database transaction when creating job + enqueue marker if possible
- Make workers idempotent
- Store generated files in object storage, not local disk
- Time out rendering jobs
- Put limits on input size and HTML complexity
- Keep template versioning so regenerated PDFs are reproducible
10) Monitoring and observability
Track:
- job success rate
- retry count distribution
- average generation duration
- webhook delivery success rate
- failure reasons
- queue lag
Add logs with:
job_idattemptevent_idtenant_id
This makes debugging much easier.
11) Minimal implementation outline
Components
- API server
- Job DB table
- Queue
- PDF worker
- Webhook delivery worker
- Webhook delivery table/log
- Object storage
Pseudocode
def create_pdf_job(payload, idempotency_key):
job = find_existing_job(idempotency_key)
if job:
return job
job = insert_job(status="queued", payload=payload, idempotency_key=idempotency_key)
queue.send({"job_id": job.id})
return job
def process_job(job_id):
job = load_job(job_id)
mark_processing(job)
try:
pdf_bytes = render_pdf(job.payload)
url = upload_pdf(pdf_bytes, job.id)
mark_succeeded(job, url)
enqueue_webhook("pdf.job.completed", job)
except TransientError as e:
schedule_retry(job, e)
except PermanentError as e:
mark_failed(job, e)
enqueue_webhook("pdf.job.failed", job)
def deliver_webhook(event):
try:
send_signed_webhook(event.target_url, event.payload)
mark_webhook_delivered(event)
except RetryableWebhookError:
retry_later(event)
12) If you want a simple stack
A common setup:
- API: Node.js / Python / Go
- Queue: SQS or RabbitMQ
- Worker: same language as API
- PDF rendering: Chromium/Puppeteer, wkhtmltopdf, or a library depending on your content
- Storage: S3
- DB: Postgres
- Webhook retries: separate queue + worker
If you want, I can also provide:
- a reference architecture diagram,
- a Node.js example with BullMQ,
- a Python example with Celery, or
- a production-ready webhook retry pattern.