How do I build an internal search box for company docs with source links and citations?

To build an internal search box for company docs with source links and citations, think of it as search + retrieval + answer generation + provenance.

1) Core product behavior

When a user searches, the system should:

1. Find the most relevant documents/passages.
2. Generate an answer from those sources.
3. Attach citations that point to the exact source passages.
4. Provide source links so users can open the original doc.

A good UX pattern is:

• Search bar at top
• Results panel with:
  • short answer
  • cited snippets
  • list of source documents
• Clicking a citation opens the exact document section or highlights the passage

---

2) Recommended architecture

A standard implementation has 5 pieces:

A. Document ingestion

Pull documents from:

• Google Drive / Drive folders
• Confluence / Notion / SharePoint
• PDFs, Word docs, HTML pages
• Internal wikis or databases

For each document:

• extract text
• preserve metadata:
  • title
  • URL
  • author
  • updated_at
  • access permissions
  • section headings
  • page numbers / paragraph IDs if available

B. Chunking

Split documents into searchable passages, usually:

• 200–500 tokens per chunk
• overlap 10–20% between chunks

Keep metadata per chunk:

• doc_id
• chunk_id
• source_url
• page/heading/anchor
• permissions

This is what citations will reference.

C. Indexing

Create a search index using one or both:

• keyword search: Elasticsearch/OpenSearch
• semantic search: vector database like pgvector, Pinecone, Weaviate, Milvus, or FAISS

Best practice is hybrid search:

• keyword search for exact terms, names, IDs
• vector search for semantic meaning

D. Retrieval + reranking

At query time:

1. run keyword + vector search
2. merge candidates
3. rerank the top results with a cross-encoder or LLM-based scorer
4. select the best passages for answer generation

E. Answer generation with citations

Use an LLM with a strict prompt:

• answer only from retrieved chunks
• cite every factual claim
• if evidence is insufficient, say so

Have the model output structured citations like:

• [1]
• [2]

Then map each citation to a source chunk and URL.

---

3) How citations should work

You want citations that are:

• traceable to a passage
• clickable
• stable over time

Citation format

Example:

"Vacation accrual begins after 90 days of employment." [1]

Where [1] corresponds to:

• document title
• URL
• page/section
• passage text

Example citation object:

{
  "id": 1,
  "title": "HR Policy Handbook",
  "url": "https://intranet/docs/hr-policy",
  "page": 14,
  "section": "Time Off",
  "snippet": "Vacation accrual begins after 90 days of employment."
}

Better UX

Show:

• citation marker in answer
• source card underneath
• “Open source” link
• highlight quoted text in the doc viewer

---

4) Security and access control

This is critical for internal docs.

You must enforce:

• document-level permissions
• row-level permissions on chunks
• user authentication
• audit logging

Rules:

• users should only retrieve chunks they are allowed to access
• search results must be filtered by ACL before answer generation
• citations should never reveal restricted docs

If you use a vector DB, store ACL metadata in the index and filter at query time.

---

5) Practical implementation stack

A common stack:

Ingestion

• Python
• unstructured.io / Apache Tika / custom parsers
• connectors to Drive/Confluence/SharePoint

Search/index

• Elasticsearch/OpenSearch for keyword
• Postgres + pgvector or Pinecone for embeddings

Orchestration

• FastAPI / Node.js backend
• queue workers: Celery, Sidekiq, BullMQ, or Kafka consumers

LLM layer

• OpenAI / Azure OpenAI / Anthropic / self-hosted model
• use structured output or function calling for citations

Frontend

• React / Next.js
• search input
• result cards
• doc preview with highlighted citations

---

6) A simple retrieval flow

1. User asks: “What is our remote work policy?”
2. Backend authenticates user
3. Backend searches allowed chunks
4. Retrieves top 20 candidates
5. Reranks to top 5
6. Sends top 5 passages to LLM
7. LLM returns:
   • answer
   • citations tied to chunk IDs
8. Frontend renders answer and clickable citations

---

7) Prompting strategy

Use a prompt like:

You are answering questions using only the provided sources.
Cite every factual statement with source IDs.
If the sources do not contain enough information, say you cannot determine it.
Do not invent details.

And provide source chunks in a structured format:

[1] Title: HR Policy Handbook
URL: ...
Text: ...
[2] Title: Remote Work Guidelines
URL: ...
Text: ...

Then request JSON output:

{
  "answer": "...",
  "citations": [
    {"source_id": 1, "quote": "..."},
    {"source_id": 2, "quote": "..."}
  ]
}

---

8) Handling source links

Each chunk should preserve the original source location:

• web docs: URL + anchor
• PDFs: page number
• Notion/Confluence: page ID + section anchor
• Office docs: link to file and optionally the paragraph if supported

If the source system allows it, create deep links to:

• page anchors
• text selection
• search highlight query

---

9) Evaluation

Measure:

• retrieval precision/recall
• citation correctness
• answer faithfulness
• permission leakage
• latency

Have humans check:

• are citations actually supporting claims?
• does the link open the right place?
• are restricted docs excluded?

---

10) Best practices

• Use hybrid search
• Keep chunks small enough for precision
• Store metadata and ACLs with each chunk
• Require citations for every answer
• Return “I couldn’t find that” when evidence is weak
• Log query, retrieved sources, and output for debugging
• Reindex documents on updates

---

11) Minimal MVP

If you want the fastest path:

1. ingest docs into chunks
2. store chunks in Postgres + pgvector
3. add keyword search with Postgres full-text or OpenSearch
4. retrieve top passages
5. use an LLM to generate an answer with citations
6. render source links in a web UI

That’s enough for a solid first version.

---

If you want, I can also give you:

• a reference architecture diagram
• a database schema for docs/chunks/citations
• or a sample FastAPI + pgvector implementation