Prompt
How do I set up retries for failed jobs in a queue?
Latest observation
Oct 24, 2026
OpenAI APIWeb search: off
To set up retries for failed jobs in a queue, the exact steps depend on the queue system you’re using, but the general pattern is the same:
Common retry strategy
-
Detect failure
- Mark a job as failed if it throws an error, times out, or returns an explicit failure.
-
Track retry metadata Store fields like:
attemptsorretry_countmax_attemptsnext_retry_atlast_errorstatus(queued,running,failed,dead)
-
Requeue with delay
- On failure, increment the attempt count.
- If
attempts < max_attempts, put the job back into the queue, often with a backoff delay. - If not, move it to a dead-letter queue or mark it permanently failed.
-
Use backoff Common options:
- Fixed delay: retry every 30 seconds
- Exponential backoff: 1 min, 2 min, 4 min, 8 min
- Jitter: randomize delay slightly to avoid retry storms
-
Make jobs idempotent
- Retries can run the same job more than once, so the job should be safe to repeat without causing duplicate side effects.
Example logic
Pseudo-code:
def process_job(job):
try:
do_work(job)
mark_complete(job)
except Exception as e:
job.attempts += 1
job.last_error = str(e)
if job.attempts < job.max_attempts:
job.next_retry_at = now() + backoff(job.attempts)
requeue(job)
else:
move_to_dead_letter_queue(job)
If you’re using a specific system
Different systems have built-in retry settings:
- Celery:
autoretry_for,retry_backoff,max_retries - Sidekiq: built-in retry mechanism with configurable retry count and backoff
- BullMQ / Bull:
attempts,backoff - AWS SQS: retry via visibility timeout + redrive policy to DLQ
- RabbitMQ: retry with dead-letter exchanges/queues and delayed requeue
If you tell me which queue library or platform you’re using, I can show the exact configuration and code.