How do I keep a RAG bot updated when our source docs change all the time?

To keep a RAG bot fresh when source docs change frequently, you want an incremental indexing pipeline, not a “rebuild everything” workflow.

Core idea

Treat your documents like a data source with:

1. Change detection
2. Targeted re-processing
3. Versioned embeddings/indexes
4. Fast deletion of stale chunks
5. Monitoring and replay

---

Recommended architecture

1) Detect changes in source docs

Use one of these patterns:

• Webhooks/events from your CMS, Drive, Confluence, Notion, Git, etc.
• Scheduled sync with file hashes or modified timestamps
• Diff-based ingestion for Git-backed docs or markdown files

Track for each document:

• document ID / URL / path
• last modified time
• content hash
• version number
• source system metadata

This lets you answer: “Did the content actually change?”

---

2) Re-chunk only changed documents

When a doc changes:

• re-fetch it
• clean/normalize it
• split into chunks again
• compute chunk-level hashes

If only part of the doc changed, you can avoid re-embedding unchanged chunks by using:

• stable chunking
• chunk hashes
• chunk IDs based on doc ID + chunk position + content hash

If the chunk content hash is unchanged, skip embedding.

---

3) Upsert new chunks, retire stale ones

Your vector store should support:

• upsert new or updated embeddings
• delete chunks that no longer exist

A common pattern:

• mark old chunks as inactive or delete them
• insert updated chunks with new IDs
• keep a document-to-chunk mapping so cleanup is easy

Example metadata:

• doc_id
• chunk_id
• chunk_hash
• source_version
• status: active/inactive
• updated_at

---

4) Keep a canonical raw-doc store

Don’t rely on embeddings as the only source of truth.

Store:

• raw document text
• parsed text
• chunked text
• embedding metadata
• source version

This helps with:

• re-indexing if your chunking strategy changes
• debugging retrieval issues
• audits and rollback

---

5) Use hybrid retrieval and metadata filters

If docs change often, hybrid retrieval helps:

• vector search for semantic matches
• keyword/BM25 for exact terms, product names, error codes, dates

Also use metadata filters:

• product area
• language
• region
• doc version
• visibility / permissions

This reduces the chance of stale or irrelevant retrieval.

---

Update strategies by doc type

Fast-changing docs

Examples: support articles, release notes, policies, tickets

Use:

• webhook-driven updates
• short sync intervals
• immediate re-index on change
• aggressive deletion of stale chunks

Frequently edited but structured docs

Examples: handbooks, wikis

Use:

• hash-based incremental sync
• section-aware chunking
• version metadata
• periodic full reconciliation

Large immutable docs

Examples: PDFs, manuals, archived docs

Use:

• periodic resync
• checksum validation
• less frequent re-embedding

---

Make chunking more stable

A lot of RAG staleness comes from bad chunking, not just bad sync.

Tips:

• chunk by headings/sections when possible
• keep chunk boundaries stable across edits
• avoid tiny chunks that shift constantly
• include section titles in chunk text
• store parent/child relationships

Stable chunking means fewer unnecessary re-embeddings.

---

Handle deletions carefully

Docs often get deleted or renamed. Your pipeline should detect:

• removed files
• moved pages
• renamed URLs

Then:

• delete all associated chunks from the vector store
• remove from your raw-doc catalog or mark as tombstoned
• optionally keep version history for audit

---

Add freshness signals at query time

Even with good syncing, retrieval should prefer fresher content.

You can:

• boost recent docs
• filter out older versions
• prefer source-of-truth collections over archived collections
• show citations with timestamps/version info

If a user asks, “What’s the latest policy?”, your bot should know to favor the newest version.

---

Monitor and test

Set up checks for:

• indexing lag
• failed document syncs
• stale chunk counts
• retrieval quality on a test set
• “answer cites old doc” incidents

Useful metrics:

• time from source update → indexed
• % docs indexed successfully
• embedding throughput
• stale-result rate
• top-k retrieval accuracy on recent changes

---

Practical implementation pattern

A solid production flow looks like:

1. Source system emits change event
2. Ingestion service fetches doc
3. Normalize + hash doc
4. Compare against previous version
5. Split into chunks
6. Hash each chunk
7. Embed only new/changed chunks
8. Upsert to vector DB
9. Delete stale chunks
10. Record version + audit trail

---

If you want the simplest robust setup

Start with:

• webhook or hourly sync
• document hashes
• chunk hashes
• upsert/delete in vector DB
• raw text archive
• metadata versioning

That’s usually enough for a bot that stays current without full rebuilds.

If you want, I can also give you:

1. a reference architecture diagram,
2. a sample sync/embedding pseudocode pipeline, or
3. recommendations for specific vector DBs and doc sources.