Prompt
How do I build real-time sync for an offline-first mobile app?
Latest observation
To build real-time sync for an offline-first mobile app, design around local-first data, change capture, conflict handling, and incremental sync.
1) Make the local device the primary store
Your app should read and write to a local database first:
- SQLite / Realm / WatermelonDB / Couchbase Lite / ObjectBox, etc.
- UI always uses local data, so it works offline instantly.
- Every mutation is saved locally immediately.
2) Record changes as operations, not just final state
Instead of syncing only “current rows,” capture what changed:
- Insert/update/delete operations
- Timestamps
- Device/user ID
- Version or sequence number
- Optional “pending sync” status
This makes it easier to:
- Retry after network failure
- Merge changes
- Sync incrementally
3) Use a sync protocol with checkpoints
Have the client and server exchange:
- Client checkpoint: last server version/sequence the client has seen
- Server checkpoint: last client change the server has applied
Typical flow:
- Client sends local pending changes since last sync.
- Server applies them and returns accepted/rejected results.
- Client requests remote changes since its last checkpoint.
- Client applies them locally.
- Both sides update checkpoints.
This lets you do delta sync instead of full downloads.
4) Choose a conflict resolution strategy
Real-time sync always needs conflict handling. Common options:
Simple approaches
- Last write wins (LWW): easiest, but can overwrite data silently.
- Server wins: server is authoritative, client changes may be discarded.
- Client wins: useful for some personal data, but risky.
Better approaches
- Field-level merge: merge independent fields separately.
- Per-entity merge rules: custom logic per object type.
- Operational transforms / CRDTs: best for collaborative editing, but more complex.
For many mobile apps, a hybrid works best:
- Use LWW for non-critical fields
- Use custom merge rules for important entities
- Surface conflicts to the user when necessary
5) Push updates in real time when connected
For “real-time” behavior, use a server push channel:
- WebSockets
- Firebase Realtime Database / Firestore listeners
- GraphQL subscriptions
- MQTT
- SSE (less common on mobile, but possible)
Pattern:
- Client syncs via HTTP/REST for reliability
- Server pushes notifications of new changes via WebSocket/push
- Client then fetches the actual deltas if needed
This avoids relying solely on push for data transfer.
6) Design for idempotency and retries
Mobile networks are unreliable, so every sync call should be safe to retry:
- Give each operation a unique ID
- Server ignores duplicates
- Client can resend pending operations after reconnect
- Use exponential backoff
This prevents duplicate writes when requests are repeated.
7) Handle deletions explicitly
Deletes need special treatment:
- Use tombstones or delete markers
- Keep deleted IDs long enough to sync to other devices
- Don’t immediately hard-delete if other devices might still need the deletion
8) Keep the payloads small
For performance:
- Sync only changed records
- Paginate large backlogs
- Compress payloads
- Send summaries first, details later if needed
- Use binary protocols only if necessary; JSON is often fine to start
9) Build a robust sync state machine
Your app should know states like:
- Idle
- Syncing
- Offline
- Conflict
- Retry pending
- Auth expired
This helps with:
- UI indicators
- Error recovery
- Avoiding overlapping sync jobs
10) Add observability
Sync bugs are hard to debug, so log:
- Sync attempt IDs
- Checkpoints
- Number of changes sent/received
- Conflict counts
- Retry reasons
- Latency and failures
A practical architecture
A common setup is:
On device
- UI
- Local DB
- Change queue/outbox
- Sync engine
On server
- API for pushing local changes
- API for fetching remote deltas
- Real-time notification channel
- Conflict resolver
- Change log / event store
Recommended sync pattern: Outbox + Pull deltas
This is often the simplest reliable model:
- User edits data offline.
- App writes to local DB and appends to an outbox table.
- On reconnect, app sends outbox items to server.
- Server stores/applies changes and records them in a change log.
- App pulls all server changes since its last cursor.
- App clears successfully synced outbox items.
When to use CRDTs
Use CRDTs if:
- Multiple users edit the same document simultaneously
- You need near-instant collaboration
- Conflicts must be mathematically mergeable
Examples:
- Text editors
- Whiteboards
- Collaborative task boards
For typical CRUD apps, CRDTs are often overkill.
A good starting stack
If you want to implement this yourself:
- Local DB: SQLite
- Sync queue: outbox table
- API: REST
- Real-time notifications: WebSocket
- Conflict policy: versioning + field merge + manual conflict UI for edge cases
If you want a managed option:
- Firebase Firestore
- Realm Sync
- Couchbase Mobile
- Supabase with custom sync logic
Minimal sync algorithm
High-level:
- Save changes locally.
- Add mutation to outbox.
- When online:
- Send outbox mutations with IDs and base versions
- Server validates and applies
- Client fetches changes since cursor
- Merge locally
- Update cursor and clear acknowledged outbox items
If you want, I can also give you:
- a reference architecture diagram,
- a sample sync protocol design, or
- a code example in React Native, Flutter, Kotlin, or Swift.