Prompt
Why are my eval scores fine but users still say the answers are bad?
Latest observation
This is very common. “Good evals” and “good user experience” often measure different things.
Why this happens
1) Your evals may be measuring the wrong thing
Offline metrics often reward:
- factual correctness on a fixed dataset
- exact match / BLEU / ROUGE-style overlap
- task completion on clean prompts
But users care about:
- usefulness
- tone
- relevance to their specific context
- whether the answer actually helps them move forward
- whether it feels confident, concise, and trustworthy
So a model can score well on evals while still feeling frustrating in practice.
2) Eval data is usually cleaner than real user traffic
Benchmarks and test sets tend to be:
- well-formed
- unambiguous
- short
- representative only of the “easy” cases
Real users ask messy things:
- vague questions
- multi-part requests
- domain-specific edge cases
- incomplete context
- contradictory constraints
If your eval set doesn’t reflect that distribution, scores will look better than reality.
3) The model may be optimizing the metric, not the experience
A system can learn to:
- answer “safely” but unhelpfully
- give generic responses that pass rubric checks
- over-explain when users want an action
- under-explain when users need confidence
- avoid mistakes by being vague
That can look good in evals and bad to users.
4) Small failures are very visible to users
Even if 80–90% of responses are fine, users tend to remember:
- one wrong answer
- one hallucination
- one annoying refusal
- one off-tone response
A few high-severity failures can dominate perception.
5) The user experience includes more than answer quality
Users judge:
- latency
- formatting
- conversational flow
- whether the assistant asks good clarifying questions
- whether it follows instructions
- whether it remembers context
- whether it adapts to the user’s level
If those are off, the answer can be “technically correct” but still bad.
What to do about it
1) Inspect real failures, not just aggregate scores
Sample conversations from production and categorize failures:
- wrong answer
- incomplete answer
- irrelevant answer
- too verbose / too terse
- bad tone
- hallucination
- refusal when it should answer
- failure to ask clarifying questions
You’ll usually find a few dominant failure modes.
2) Build evals from real traffic
Create a test set from:
- recent user queries
- failed sessions
- hard edge cases
- high-value tasks
This makes your evals more representative of what users actually experience.
3) Use human preference evals
Ask humans:
- Which answer is more helpful?
- Which answer would you send to a user?
- Which answer resolves the issue faster?
Preference judgments often correlate better with user satisfaction than single-number metrics.
4) Add task-specific quality checks
Examples:
- Did the response answer the actual question?
- Did it use the right level of detail?
- Did it ask for clarification when needed?
- Did it avoid unsupported claims?
- Did it provide an actionable next step?
A rubric like this is often more useful than a generic score.
5) Measure downstream outcomes
If possible, track:
- user re-asks
- abandonment rate
- thumbs down
- escalation to human support
- task completion rate
- time to resolution
These are closer to what users feel than offline evals.
6) Look at calibration and uncertainty
Sometimes users dislike answers because the model sounds too confident when it’s unsure. Helpful behaviors include:
- stating uncertainty
- asking a clarifying question
- offering options
- explaining assumptions
A useful mental model
Think of evaluation in three layers:
- Capability — can the model produce a correct answer?
- Behavior — does it answer in a useful, safe, and appropriate way?
- Experience — does the user feel helped, understood, and confident?
Many teams evaluate layer 1 well, but users complain about layer 2 or 3.
If you want, I can help you diagnose this for your setup by suggesting:
- an eval audit checklist
- a production failure taxonomy
- or a rubric that better predicts user satisfaction.