Prompt
I'm building a Python app with background tasks and retries — what should I use?
Latest observation
Oct 24, 2026
OpenAI APIWeb search: off
If you’re building a Python app that needs background jobs, retries, scheduling, and reliable task execution, the best choice usually depends on how much infrastructure you want to manage.
Good options
1. Celery — best for mature, distributed task processing
Use this if you need:
- Background jobs running outside the web request
- Retries with backoff
- Task queues
- Multiple workers/processes/machines
- Integrations with Redis or RabbitMQ
Pros:
- Very widely used and battle-tested
- Great retry support
- Scales well
- Supports scheduled jobs with Celery Beat
Cons:
- More setup and operational complexity
- Can be overkill for small apps
Typical stack: Celery + Redis or Celery + RabbitMQ
2. RQ (Redis Queue) — simpler than Celery
Use this if you want:
- Basic background jobs
- Easy setup
- Redis-backed queue
- Simpler codebase
Pros:
- Easier to get started than Celery
- Good for straightforward background work
Cons:
- Fewer features than Celery
- Less powerful scheduling and routing options
3. Dramatiq — modern alternative with a clean API
Use this if you want:
- Background jobs with retries
- Cleaner developer experience
- Less complexity than Celery
Pros:
- Nice API
- Built-in retries
- Good balance of simplicity and power
Cons:
- Smaller ecosystem than Celery
4. Huey — lightweight task queue
Use this if you need:
- Small-to-medium app
- Simple background tasks
- Periodic jobs
Pros:
- Lightweight
- Easy to use
- Supports retries and scheduling
Cons:
- Less common than Celery
- Not ideal for very large systems
Quick recommendation
- Small app / simple jobs:
HueyorRQ - Need retries + serious scalability:
Celery - Want something modern and simpler than Celery:
Dramatiq
If you’re using FastAPI or Django
- FastAPI: Celery or Dramatiq are common choices
- Django: Celery is the most established option
My practical default
If you’re unsure and expect the app to grow, I’d pick:
Celery + Redis
because it gives you the most flexibility for retries, workers, scheduling, and long-term growth.
If you want, I can also give you:
- a comparison table,
- a recommended stack for FastAPI/Django, or
- a minimal working example for Celery retries.