What should I use for a chatbot that needs citations from source docs?

For a chatbot that needs citations from source documents, the best general choice is:

Use RAG (Retrieval-Augmented Generation) with chunked documents and source tracking

That means:

1. Store your docs in a searchable index

   • Usually a vector database (for semantic search)
   • Often combined with keyword search too

2. Retrieve the most relevant passages for each user question

3. Generate the answer only from those passages

4. Attach citations by keeping metadata on each chunk:

   • document title
   • page number
   • section / heading
   • URL or file name
   • chunk ID

Good options

If you want a simple, practical setup

• LlamaIndex or LangChain for RAG orchestration
• Postgres + pgvector or Pinecone / Weaviate / Qdrant for retrieval
• An LLM like GPT-4.1 / GPT-4o, Claude, or similar for answer generation

If you want citations that are easier to verify

• Use hybrid search: vector + keyword
• Retrieve top passages
• Have the model quote or paraphrase with explicit source labels like:
  • [Smith Handbook, p. 14]
  • [Policy Doc, section 3.2]

If the docs are PDFs

• Use a parser that preserves layout/page info:
  • Unstructured
  • Apache Tika
  • PDF text extraction tools with page metadata

Important design tips

• Chunk by meaning, not just fixed size
• Preserve page/section metadata
• Use strict prompting: “Only answer from provided sources. If unsupported, say you don’t know.”
• Return citations at the sentence or paragraph level if possible
• Consider a reranker to improve source quality

Best default stack

If you just want something solid and common:

• LlamaIndex
• Qdrant or pgvector
• OpenAI GPT-4.1 / GPT-4o
• Document parser that keeps page numbers
• Hybrid retrieval + reranking

If you want, I can recommend a stack based on:

• your scale,
• whether your docs are PDFs/web pages/Word files,
• and whether you need exact page citations or just document citations.