Prompt
what should I use for grounding and retrieval evals?
Latest observation
Nov 12, 2026
OpenAI APIWeb search: off
For grounding and retrieval evaluation, what you should use depends on whether you’re evaluating:
- retriever quality only,
- RAG/QA end-to-end grounding, or
- factuality / attribution of generated answers.
Here’s the practical breakdown.
1) If you want to evaluate the retriever itself
Use classic IR metrics:
- Recall@k: Did the relevant document appear in the top-k?
- MRR: How early was the first relevant result?
- nDCG@k: Good when relevance is graded, not binary
- Precision@k: How many of the retrieved docs are relevant?
Best choice
- Recall@k is usually the most important for RAG, because if the right evidence isn’t retrieved, generation can’t be grounded.
- Pair it with nDCG@k or MRR if ranking quality matters.
Use when
- You have labeled relevant passages/docs for each query
- You’re comparing embedding models, chunking strategies, or index configs
2) If you want to evaluate grounding in generated answers
Use answer-groundedness / attribution metrics:
- Faithfulness: Is the answer supported by the retrieved context?
- Groundedness / supportedness: Are claims traceable to sources?
- Citation precision/recall: If your system cites sources, are the citations correct and sufficient?
- Hallucination rate: How often does the model say unsupported things?
Best choice
- Faithfulness (or equivalent groundedness score) is the most common and useful high-level metric.
- If you have citations, measure:
- Citation precision: cited sources actually support the claim
- Citation recall: supported claims are cited
Use when
- You want to know whether the model used the retrieved context
- You care about compliance, trust, or auditable responses
3) If you want end-to-end RAG eval
Use a combination of:
- Retrieval metrics: Recall@k, MRR, nDCG@k
- Grounding metrics: Faithfulness / supportedness
- Answer quality metrics:
- Exact Match / F1 for QA
- Semantic similarity / LLM judge for open-ended answers
- Task success rate
Best practice
Don’t rely on a single score. A good RAG eval bundle is:
- Context retrieval quality
- Recall@k
- Answer correctness
- EM/F1 or task-specific score
- Groundedness
- Faithfulness / citation correctness
This helps separate:
- bad retrieval,
- good retrieval but bad generation,
- good answer but unsupported answer.
Recommended tools / frameworks
If you’re asking “what should I use” in terms of libraries:
Popular options
- Ragas
Good for RAG evaluation: faithfulness, answer relevance, context precision/recall, etc. - TruLens
Good for feedback functions and grounding-style evals - Arize Phoenix
Strong for tracing, retrieval debugging, eval workflows - LangSmith
Useful for dataset-based evals and tracing - DeepEval
Convenient for custom LLM-as-judge evaluations
If you want a lightweight setup
- Compute Recall@k / MRR yourself
- Use LLM-as-judge for groundedness
- Add a small human-labeled set for calibration
What I’d recommend in practice
If you’re building a RAG system, start with:
- Retrieval: Recall@k
- Generation correctness: EM/F1 or human judgment
- Grounding: Faithfulness / supportedness
- Optional: citation precision if you cite sources
If you need one “starter stack,” use:
- Ragas for groundedness/context metrics
- Your own IR metrics for retrieval
- Human review on a small gold set for validation
Rule of thumb
- If the question is “did we fetch the right stuff?” → use Recall@k
- If the question is “did the model answer using that stuff?” → use Faithfulness
- If the question is “did the system answer correctly overall?” → use task-level answer accuracy
If you want, I can also give you:
- a metric selection table for different RAG setups, or
- a minimal evaluation pipeline with exact metrics and thresholds.