Prompt
How do I measure brand visibility in LLM responses?
Latest observation
Measuring brand visibility in LLM responses is basically about estimating how often, how prominently, and in what context your brand appears when people ask models relevant questions.
Here’s a practical way to do it.
1) Define what “visibility” means
You usually want to measure a few different things:
- Mention rate: How often the brand is named at all.
- Share of voice: Of all brands mentioned in a category, what share is yours?
- Ranking position: If the model lists options, where does your brand appear?
- Sentiment / framing: Is the brand presented positively, neutrally, or negatively?
- Recommendation rate: How often does the model recommend your brand versus competitors?
- Citation presence: If the model uses sources, is your brand/source cited?
2) Build a query set
Create a representative set of prompts that users might ask.
Example categories:
- Category discovery: “What are the best project management tools for small teams?”
- Comparison: “Notion vs Asana vs ClickUp”
- Problem-solving: “What tool is best for tracking tasks across departments?”
- Branded: “Is [your brand] good for startups?”
- Intent-based: “Best budget CRM for nonprofits”
Make sure you include:
- Head terms and long-tail terms
- Beginner and expert queries
- Branded and unbranded queries
- High-intent and informational prompts
3) Run the prompts across models
Test across:
- Different model providers
- Different versions of the same model
- Different temperatures / sampling settings
- Different system instructions, if relevant
LLM outputs can vary, so you want multiple runs per prompt to estimate stability.
4) Score the responses
For each response, capture:
A. Presence
- Does the brand appear? Yes/No
B. Position
- First mention, top 3, top 5, etc.
C. Context
- Is it in a recommendation list?
- Is it mentioned as an example?
- Is it contrasted with competitors?
D. Sentiment
You can score this manually or with another classifier:
- Positive
- Neutral
- Negative
E. Recommendation strength
For example:
- 0 = not mentioned
- 1 = mentioned but not recommended
- 2 = recommended with caveats
- 3 = strongly recommended
5) Calculate metrics
Useful metrics include:
- Brand mention rate = responses mentioning brand / total responses
- Top-k visibility = responses where brand appears in top 3 or top 5 / total responses
- Share of voice = brand mentions / all competitor mentions
- Average rank = average position when mentioned
- Sentiment-weighted visibility = visibility score adjusted by sentiment
- Category coverage = number of query types where brand appears
You can also create a composite LLM Visibility Index like:
Visibility = (mention rate × 0.3) + (top-3 rate × 0.3) + (positive framing rate × 0.2) + (recommendation rate × 0.2)
Adjust weights based on what matters to you.
6) Benchmark against competitors
Visibility is only meaningful relative to others. Compare:
- Your brand vs key competitors
- Across query types
- Across models
- Over time
This helps identify whether you’re weak on:
- Discovery prompts
- Comparison prompts
- “Best X” prompts
- Niche use cases
7) Track over time
LLM visibility changes as:
- models are updated
- retrieval/indexing changes
- new content gets published
- citations and rankings shift
Track weekly or monthly and look for trends.
8) Add qualitative review
Numbers alone can hide important nuances. Review sample outputs to understand:
- Why the brand is included/excluded
- Whether the model is relying on outdated info
- Whether competitors are being favored due to better web presence
- Whether the model is hallucinating brand capabilities
9) Automate with a pipeline
A simple pipeline looks like:
- Generate prompt set
- Query model(s)
- Parse outputs for brand names
- Score mentions, rank, sentiment
- Store results in a database or spreadsheet
- Visualize trends in a dashboard
10) Be careful about limitations
LLM visibility is not the same as:
- SEO rankings
- web traffic
- purchase intent
- true market share
Also note:
- Models can be inconsistent
- Some models use retrieval and others don’t
- Outputs vary by phrasing
- Brand names can be ambiguous
Practical starter metric set
If you want a simple first version, track these three:
- Mention rate
- Top-3 appearance rate
- Positive recommendation rate
That’s usually enough to get started and see meaningful patterns.
If you want, I can also give you:
- a spreadsheet template
- a scoring rubric
- a Python script to automate brand visibility measurement