Prompt
What should I use for background jobs in a web app?
Latest observation
Nov 9, 2026
OpenAI APIWeb search: off
It depends on what kind of background jobs you need, but here’s the practical rule of thumb:
Good default choices
1) Simple app / moderate workload
Use a job queue + worker process:
- Python: Celery + Redis/RabbitMQ
- Node.js: BullMQ + Redis
- Ruby: Sidekiq + Redis
- PHP/Laravel: Laravel Queues + Redis/database
- Django: Celery or Django-Q
This is the standard choice for:
- sending emails
- processing images/files
- generating reports
- syncing data
- retrying failed tasks
- scheduled tasks
2) Small app, low volume
If jobs are lightweight and not critical, you can start with:
- a database-backed queue
- or even a simple cron job + script
This is fine for:
- nightly cleanup
- periodic imports
- small internal tools
3) High scale / reliability / complex workflows
Consider managed or more robust systems:
- AWS SQS + worker service
- GCP Cloud Tasks / Pub/Sub
- Azure Queue Storage / Service Bus
- Temporal for complex workflows that need retries, state, and orchestration
Use these if you need:
- very high reliability
- horizontal scaling
- delayed jobs
- workflow orchestration
- long-running business processes
Quick recommendations by situation
- Need the easiest common stack? Redis + BullMQ/Celery/Sidekiq
- Need strong durability and managed infra? SQS/Cloud Tasks
- Need workflow orchestration, not just jobs? Temporal
- Only occasional scheduled tasks? cron
What to avoid
- Doing jobs inside the web request
- Relying on an in-memory queue in the web process
- Using cron for everything if you need retries, monitoring, or many concurrent jobs
My practical recommendation
For most web apps:
- Use a queue with separate workers
- Use Redis if you want simplicity
- Use SQS or similar managed queues if you want better operational safety
If you tell me your stack (language/framework), job types, and expected volume, I can recommend a specific setup.