We have docs scattered across Confluence, email, and tickets. What’s the best way to build search so users can find answers without knowing exact file names?

The best approach is semantic search with strong indexing and answer retrieval, not just keyword search.

What to build

1) Ingest all sources into one searchable index

Pull content from:

• Confluence pages
• Email threads
• Ticket systems like Jira/Zendesk/ServiceNow

Normalize each item into a common format:

• title
• body text
• author
• date
• source
• permissions/ACLs
• tags/project/customer references

This lets users search across systems without caring where the answer lives.

2) Use hybrid search

Combine:

• Keyword search for exact terms, IDs, error codes, names
• Vector/semantic search for meaning and paraphrased questions

This is important because users often ask:

• “How do I reset VPN access?”
• “What’s the process for onboarding a contractor?”
• “Who approved this exception last quarter?”

Those won’t always match file names or exact phrasing.

3) Chunk documents intelligently

Don’t index huge pages as one blob. Split content into useful chunks:

• section headings
• FAQ entries
• ticket resolution notes
• email thread summaries

Keep chunk metadata so results are understandable and attributable.

4) Add a question-answer layer

After retrieval, use an LLM to:

• summarize the most relevant sources
• extract the direct answer
• cite the source snippets

This improves usability a lot: users get an answer, not just a list of links.

5) Respect permissions

Search must enforce source permissions so users only see content they’re allowed to access. This is critical for email and tickets.

6) Improve with metadata and synonyms

Add business-specific metadata:

• department
• product line
• customer
• incident type
• project

Also maintain a synonym dictionary:

• “SSO” = “single sign-on”
• “VPN” = “remote access”
• “offboarding” = “termination process”

This helps query understanding.

---

Recommended architecture

Connectors → Normalizer → Chunker → Indexer

• Keyword index (e.g. Elasticsearch/OpenSearch)
• Vector index (embeddings)
• Permission filter layer
• Retrieval + reranking
• Optional LLM answer generation

A practical search flow:

1. User asks a question
2. System expands/sanitizes query
3. Retrieve top results from keyword + vector indexes
4. Rerank by relevance, recency, source authority
5. Return snippets or a synthesized answer with citations

---

What makes it “best”

For scattered docs, the best search experience usually has:

• one unified search box
• semantic understanding
• source citations
• permission-aware results
• hybrid retrieval
• ongoing feedback loop

---

If you want a fast MVP

Start with:

1. Index Confluence + tickets first
2. Use hybrid search
3. Chunk by heading/section
4. Add ACL filtering
5. Show quoted snippets and links
6. Track “no result” and clicked results to improve relevance

Email can come later because it’s noisier and harder to normalize.

If you want, I can sketch a reference architecture, or suggest specific tools/stack options for building this.