What should I use for fresh indexing of docs and tickets into RAG?

For fresh indexing of docs and tickets into a RAG system, the best choice is usually a hybrid ingestion + incremental indexing pipeline, not a single database or vector store by itself.

Recommended setup

1) Source of truth

Keep raw docs/tickets in their original systems:

• Docs: Confluence, Google Drive, Notion, SharePoint, Git, etc.
• Tickets: Zendesk, Jira Service Management, Intercom, Salesforce, etc.

2) Ingestion layer

Use a connector or sync process that supports:

• Webhooks for near-real-time updates
• Incremental polling with updated_at, version IDs, or change tokens
• Backfill for initial load
• Deletes/tombstones so removed content is removed from the index

Good patterns:

• Event-driven if your systems support it
• Scheduled incremental sync if they don’t
• Queue-based processing for reliability

3) Document processing

Before indexing:

• Normalize text
• Split into chunks
• Preserve metadata:
  • source
  • document/ticket ID
  • timestamps
  • author/assignee
  • permissions/ACLs
  • status
  • thread/message IDs
• Deduplicate content
• Optionally enrich with titles, tags, summaries

4) Indexing layer

For RAG, use:

• Vector index for semantic search
• Keyword / BM25 index for exact match
• Prefer hybrid retrieval

Common choices:

• Vector DBs: Pinecone, Weaviate, Milvus, Qdrant, pgvector
• Search engines with vector + keyword: Elasticsearch, OpenSearch

5) Freshness strategy

To keep the index fresh:

• Store a last_synced cursor
• Re-index only changed items
• Use upserts keyed by stable IDs
• Delete chunks when the source item is deleted or superseded
• Use versioning if tickets/docs mutate frequently
• Re-embed only when content changes materially

What I’d use in practice

If you want simplest reliable architecture

• Ingestion: Airbyte, Fivetran, Meltano, custom sync jobs
• Processing: Python workers or serverless jobs
• Index: OpenSearch/Elasticsearch or Qdrant + separate keyword store
• Queue: SQS, Pub/Sub, Kafka, or Redis Streams

If you want near-real-time freshness

• Webhooks from doc/ticket systems
• Queue events
• Async workers to chunk/embed/upsert
• Periodic reconciliation job to catch missed updates

If you want minimal ops

• Managed connector + managed vector DB
• Example:
  • Airbyte + Pinecone
  • Fivetran + OpenSearch
  • Custom webhook sync + Qdrant Cloud

Best practice for tickets specifically

Tickets are often:

• conversational
• mutable
• threaded
• permission-sensitive

So index them as:

• one ticket-level “parent” record
• multiple chunked message records
• metadata with status, priority, product area, timestamps
• final resolution/summary field if available

Short answer

Use a change-driven incremental ingestion pipeline that writes to a hybrid search index (vector + keyword), with upserts, versioning, and deletions. That’s the most robust way to keep docs and tickets fresh in RAG.

If you want, I can also recommend a specific stack based on your environment:

• AWS / Azure / GCP
• Open source only
• small startup
• enterprise / compliance-heavy