Prompt
How do I audit whether our brand is represented accurately in AI-generated answers?
Latest observation
To audit whether your brand is being represented accurately in AI-generated answers, treat it like a media monitoring + SEO + QA program for LLMs.
1) Define what “accurate” means for your brand
Create a source-of-truth document covering:
- Product names and descriptions
- Pricing and packaging rules
- Key differentiators
- Company facts
- Compliance/legal claims
- Brand voice and positioning
- “Do not say” list
- Approved answers for common questions
This becomes your benchmark for judging AI responses.
2) Build a list of prompts people actually ask
Group prompts by intent:
- Awareness: “What is [brand]?”
- Comparison: “[brand] vs [competitor]”
- Consideration: “Is [brand] good for [use case]?”
- Support: “How do I return/cancel/use [brand]?”
- Trust: “Is [brand] legit/safe/scam?”
- Category: “Best [category] brands”
Include:
- Short prompts
- Long natural-language prompts
- Ambiguous prompts
- Slang/misspellings
- Region-specific prompts
- Negative prompts
3) Test across multiple AI systems
Don’t rely on one model. Check:
- ChatGPT
- Claude
- Gemini
- Perplexity
- Copilot
- Search-integrated AI results
Run the same prompt set across all of them, ideally:
- Logged with date/time
- Same account settings where possible
- Same locale/language
- Fresh sessions or clearly documented context
4) Score responses with a simple rubric
For each answer, rate:
- Correctness: Are facts right?
- Completeness: Important details missing?
- Positioning accuracy: Is the brand described fairly?
- Safety/compliance: Any risky or prohibited claims?
- Competitor accuracy: Are comparisons fair and current?
- Tone: Does it sound aligned with brand voice?
A simple 1–5 scale works well, plus notes.
5) Track the main failure types
Common issues include:
- Hallucinated product features
- Outdated pricing or policy info
- Mixing up your brand with competitors
- Overstating claims
- Understating differentiators
- Inconsistent category placement
- Bad sentiment from stale or low-quality sources
- Incomplete answers that miss your core message
6) Check the sources AI is using
For search-connected systems, review cited sources:
- Is the model pulling from your website, help center, press releases, or third-party sites?
- Are there outdated pages ranking highly?
- Are review sites or forums dominating the narrative?
- Are structured data and metadata helping or hurting?
This helps you identify whether the problem is model behavior or source content quality.
7) Compare against your own content ecosystem
Audit:
- Homepage
- Product pages
- FAQ/help docs
- Blog content
- Knowledge base
- App store / marketplace listings
- Wikipedia/Wikidata if relevant
- Review platforms
- Social profiles
- Third-party articles and directories
AI often summarizes what it finds across these sources.
8) Monitor at scale over time
Set up recurring checks:
- Weekly or monthly prompt runs
- Automated screenshot/log capture
- Trend dashboards
- Alerts for high-risk prompts or inaccurate claims
Measure:
- Accuracy rate
- Brand mention frequency
- Sentiment
- Competitor share of voice
- Citation/source quality
- Drift over time
9) Prioritize fixes by impact
Fix the highest-risk issues first:
- Legal/compliance inaccuracies
- Pricing and policy errors
- Safety claims
- Common support questions
- High-traffic comparison queries
Then address broader narrative issues.
10) Improve the inputs AI is learning from
To reduce bad answers:
- Update your website and help content
- Make key facts easy to parse
- Use consistent naming
- Publish clear FAQ pages
- Add structured data where appropriate
- Refresh stale third-party listings
- Seed authoritative, crawlable content that reflects the truth
11) Create an escalation workflow
When you find a bad answer:
- Record the prompt, model, date, and output
- Note the error type
- Identify likely source(s)
- Assign owner: legal, product, marketing, support, PR, web
- Decide whether to fix source content, request a correction, or both
12) Use a repeatable audit template
A practical row format:
- Model
- Prompt
- Date
- Response
- Correct? (Y/N)
- Error type
- Severity
- Source/citation
- Action needed
- Owner
- Status
A good baseline audit setup
If you want something lean but effective:
- 30–50 key prompts
- 3–5 AI systems
- Monthly review
- 5-point accuracy rubric
- Logged results in a spreadsheet or dashboard
If you want, I can also give you:
- a brand AI audit spreadsheet template,
- a prompt list you can reuse, or
- a scoring rubric for accuracy and risk.