Prompt
ChatGPT, I need to send transactional push notifications from events, with retries and token cleanup. Recommend a setup.
Latest observation
A solid setup is:
1) Use an event-driven notification pipeline
Have your app emit domain events like:
order_paidpassword_reset_requestedshipment_delayed
Then route those events into a notification worker/service that is responsible only for:
- building the push payload
- sending it
- handling retries
- deleting invalid tokens
This keeps your app fast and makes notifications reliable.
2) Put events onto a durable queue
Use a queue or stream such as:
- AWS SQS
- RabbitMQ
- Kafka
- Redis queue / BullMQ
- Sidekiq / Celery / Hangfire equivalents
Why:
- prevents losing notifications if push provider is down
- lets you retry safely
- smooths bursts of traffic
3) Store device tokens separately
Maintain a table/collection like:
user_iddevice_idpush_tokenplatformlast_seen_atrevoked_atdisabled_reason
A user can have multiple devices/tokens.
Also keep token metadata so you can expire old or invalid tokens cleanly.
4) Worker flow for each notification
When a worker receives an event:
- Load recipient(s)
- Fetch active tokens
- Send push to the provider
- Inspect the response
- Mark success/failure per token
- Retry only transient failures
- Delete or disable invalid tokens
5) Retry policy
Retry only for temporary issues, like:
- network timeouts
- 5xx from provider
- rate limiting
Use:
- exponential backoff
- jitter
- max attempts, e.g. 3–5
- dead-letter queue for exhausted jobs
Do not retry permanent errors such as:
- invalid token
- unregistered token
- malformed payload
- unauthorized credentials
6) Token cleanup rules
Automatically remove or disable tokens when the provider says they’re invalid.
Examples:
- FCM:
UNREGISTERED,INVALID_ARGUMENTfor bad registration token - APNs:
Unregistered,BadDeviceToken - SNS / Expo / OneSignal: provider-specific “invalid token” / “device not registered” type errors
Best practice:
- soft-disable on first invalid response
- hard-delete after confirmation or repeated invalidation
- periodically prune stale tokens not seen in a long time
7) Ensure idempotency
Notification jobs may run more than once. Add an idempotency key such as:
event_id + recipient_id + notification_type
Store “sent” status so retries don’t duplicate transactional pushes.
8) Observability
Track:
- queued jobs
- sent count
- retry count
- failed count
- invalid token count
- delivery latency
Add logs with:
- event id
- user id
- device token id
- provider response code
This helps debugging and token hygiene.
9) Recommended architecture
A practical layout:
- Application service emits event
- Message queue buffers event
- Notification worker processes event
- Push provider adapter sends via FCM/APNs
- Token store manages devices
- Cleanup job periodically removes stale/invalid tokens
10) Suggested implementation choices
If you want a straightforward, robust setup:
Simple and good
- Queue: RabbitMQ or SQS
- Worker: your preferred job runner
- Push provider: FCM for Android/Web, APNs for iOS
- Token cleanup: worker updates token records from provider errors
If you want a managed push abstraction
- OneSignal, Firebase Cloud Messaging, or AWS SNS
- Still keep your own token/user mapping and idempotency
Managed services reduce integration work, but you still need retry logic and cleanup rules.
11) Practical recommendation
For transactional push notifications, I’d recommend:
- Event outbox pattern in your app
- Queue-based notification worker
- Per-token send result tracking
- Exponential backoff retries
- Automatic token revocation on invalid-token errors
- Dead-letter queue
- Daily stale-token cleanup job
If you want, I can also give you:
- a concrete architecture diagram,
- a database schema for tokens and notifications,
- or example code in Node.js / Python / Go / Ruby.