Prompt
I'm building a dashboard to monitor how ChatGPT describes our company, what should I track?
Latest observation
To monitor how ChatGPT describes your company, track a mix of accuracy, sentiment, visibility, and consistency. A good dashboard should answer: What is ChatGPT saying, how often, how well, and with what risk?
Core metrics to track
1) Brand mention rate
- How often your company is mentioned in ChatGPT responses for relevant prompts
- Split by:
- Product category
- Use case
- Competitor comparison
- General brand queries
2) Description accuracy
Measure whether ChatGPT’s description is correct on key facts, such as:
- Company name and positioning
- Products/services
- Industry
- Target customers
- Pricing or business model
- Headquarters / geography
- Founding date or company size, if relevant
You can score this as:
- Correct
- Partially correct
- Incorrect
- Missing
3) Sentiment / tone
Track whether the company is described as:
- Positive
- Neutral
- Negative
Also monitor tone dimensions like:
- Trustworthy
- Innovative
- Expensive / affordable
- Enterprise-grade / consumer-friendly
- Easy to use / complex
4) Share of voice
If users ask about your category, what portion of responses include your company versus competitors?
Useful breakdowns:
- By prompt type
- By model version
- By geography/language
- By date over time
5) Competitor comparison accuracy
When ChatGPT compares you to competitors, track:
- Whether competitors are correctly identified
- Whether feature comparisons are accurate
- Whether you are portrayed fairly
- Whether outdated competitors are being mentioned
6) Hallucination rate
How often does ChatGPT invent or distort facts about your company? Examples:
- Wrong products
- Made-up partnerships
- Incorrect pricing
- Wrong customer segment
- Confused with another company
7) Citation / source quality
If the model references sources or appears to rely on web content, track:
- Whether it cites your official site
- Whether it cites third-party sources
- Whether source quality is high or low
- Whether outdated pages are being used
8) Consistency across prompts
Does the description change depending on phrasing? Test:
- “What does [Company] do?”
- “Explain [Company] in one sentence”
- “Is [Company] good for enterprise?”
- “Compare [Company] to [Competitor]”
You want low variance in core facts.
9) Category association
What market/category does ChatGPT think you belong to?
- Correct category
- Adjacent category
- Wrong category
This matters a lot for discoverability and positioning.
10) Recommendation quality
When users ask “What should I use for X?”, does ChatGPT:
- Recommend your company?
- Mention you as an option?
- Explain fit accurately?
- Omit you entirely?
Useful dashboard views
Trend over time
- Accuracy score over time
- Mention rate over time
- Negative/positive tone over time
- Hallucination incidents over time
Prompt cluster analysis
Group prompts into buckets like:
- Brand awareness
- Product questions
- Pricing
- Competitor comparisons
- Industry/category queries
- Customer support / reviews
Top error themes
Surface the most common mistakes:
- Wrong product names
- Wrong pricing
- Wrong target market
- Incorrect competitor
- Outdated company info
Model/version comparison
Compare outputs across:
- Different ChatGPT model versions
- Different settings
- Different retrieval modes, if relevant
Recommended scoring framework
For each response, score:
- Accuracy: 0–5
- Sentiment: positive/neutral/negative
- Completeness: 0–5
- Brand alignment: 0–5
- Hallucination risk: low/medium/high
Then aggregate by prompt category and over time.
If you want a lean version
Start with just these 5:
- Mention rate
- Accuracy score
- Sentiment
- Hallucination rate
- Competitor comparison accuracy
Bonus: alerts
Set alerts for:
- Sudden drop in accuracy
- Spike in negative sentiment
- New recurring false claim
- Competitor overtaking you in mentions
- Major change after a model update
If you want, I can also help you design:
- a sample dashboard layout
- a metrics schema/table
- or a scoring rubric for evaluating responses consistently.
Brands