I’m unhappy with Elasticsearch semantic search relevance

Sorry you’re dealing with that — Elasticsearch semantic search can be very good, but relevance often disappoints unless the setup is tuned carefully.

Common reasons relevance feels off:

• Embedding mismatch: the embedding model used at indexing time doesn’t match your content/query style well.
• Chunking issues: chunks too large, too small, or split in bad places.
• No hybrid retrieval: semantic-only search can miss exact keyword intent, names, IDs, and rare terms.
• Weak reranking: top-k vector results may need a second-stage reranker.
• Field choice problems: semantic signals stored in the wrong fields or mixed with noisy text.
• Query formulation: short vague queries often need expansion or hybrid weighting.
• Index tuning: ANN/vector settings and filtering can affect recall and ranking.

A few high-impact fixes:

1. Use hybrid search

   • Combine BM25 keyword search + vector similarity.
   • This usually improves relevance dramatically for real user queries.

2. Add a reranker

   • Retrieve top 50–200 candidates, then rerank with a cross-encoder or LLM-based reranker.

3. Revisit chunking

   • Aim for semantically coherent chunks.
   • Include headings/metadata in chunks where useful.

4. Tune embeddings

   • Use a stronger model if possible.
   • Make sure query/document embeddings are produced consistently.

5. Evaluate with real queries

   • Build a small judged set of queries and expected results.
   • Measure before/after changes instead of relying on intuition.

6. Use filters and structured signals

   • If users care about date, product, language, region, etc., apply filters before ranking.

If you want, I can help in a more concrete way by doing one of these:

• diagnose your current Elasticsearch setup,
• suggest a hybrid search query,
• recommend chunking/embedding choices,
• or help build an evaluation checklist for relevance.