How do I build a system to track generative search share of voice?

To build a system for tracking generative search share of voice (SoV), think of it as measuring:

1. How often your brand appears in AI-generated answers
2. How favorably and prominently it appears
3. Compared to competitors
4. Across a set of important prompts/topics over time

Here’s a practical blueprint.

---

1) Define what “share of voice” means for generative search

In classic search, SoV often means impressions/clicks/rank visibility. In generative search, there’s no single SERP position, so you need your own definition.

A useful SoV formula:

Generative SoV = weighted brand presence in AI answers / total weighted presence across you + competitors

You can break “presence” into measurable components:

• Mentioned at all: brand appears in the answer
• Primary recommendation: brand is directly recommended as best option
• Placement prominence: early in response, bullet list, summary, cited source, etc.
• Sentiment/qualification: positive, neutral, negative
• Citation presence: brand’s site is cited or linked
• Topic relevance: answer is about a keyword/topic cluster that matters to you

Example scoring:

• Mention = 1 point
• Recommended = 3 points
• Cited = 2 points
• Top-3 placement in list = 1 point
• Positive sentiment = 1 point
• Negative mention = -1 point

Then compute:

Brand SoV = Brand score / sum of scores for all tracked brands

---

2) Decide what models and surfaces you’re tracking

Generative search can mean different surfaces:

• Google AI Overviews
• ChatGPT
• Perplexity
• Claude
• Gemini
• Bing/Copilot
• Industry-specific AI search tools

You should track by:

• Model
• Prompt
• Locale
• Device/type if relevant
• Time
• User intent category

Important: responses can vary by geography, account state, and prompt wording.

---

3) Build a prompt set that reflects demand

Your system needs a curated set of prompts rather than random ad hoc queries.

Create prompts across intent categories:

Informational

• “What is the best CRM for startups?”
• “How do I improve email deliverability?”

Commercial investigation

• “Best CRM for small business”
• “Top email tools for e-commerce”

Transactional / high-intent

• “Compare [your brand] vs [competitor]”
• “[category] pricing”

Problem/solution

• “How to reduce customer churn”
• “How to track generative search visibility”

Brand-specific

• “[your brand] reviews”
• “[your brand] vs [competitor]”

Best practice:

• Cluster keywords into topics
• Include head terms and long-tail
• Keep prompts stable over time for trend analysis
• Refresh periodically to reflect market changes

---

4) Collect answers consistently

You need a data collection layer that captures responses on a schedule.

Options

• Manual sampling for initial validation
• Automated browser-based capture
• API-based collection where allowed
• Third-party SERP/AI monitoring providers

For each prompt, store:

• Prompt text
• Model/surface
• Timestamp
• Locale
• Raw response text
• Sources/citations
• Any structured elements (lists, cards, citations)
• Screenshot or HTML when possible
• Metadata about session conditions

If using browser automation, be careful about:

• Platform terms of service
• Rate limits
• Login/cookie state differences
• Output variability

---

5) Extract brand mentions and attributes

Once you have responses, run NLP extraction to identify:

• Brand mentions
• Competitor mentions
• Product mentions
• Citations/links
• Ranking/order in lists
• Sentiment
• Claims such as “best,” “cheapest,” “most secure”

Start simple with rules, then improve with models.

Example extraction tasks

• Named entity recognition for brands
• Regex/domain matching for citations
• List parsing for ranked recommendations
• Sentiment classification for brand context
• Position detection in answer structure

Store extracted fields like:

• brand_mentioned: true/false
• brand_rank: 1/2/3/null
• brand_sentiment: positive/neutral/negative
• brand_cited: true/false

---

6) Create a scoring model

A robust SoV system usually uses weighted components.

Example:

Per prompt-answer-brand score

• Mentioned: 1
• Cited: 2
• Recommended: 3
• Top-ranked in list: 2
• Positive sentiment: 1
• Negative sentiment: -2

Then:

Brand visibility score = sum of scores across all prompt responses

Brand SoV = Brand visibility score / total visibility score of all tracked brands

You may want separate metrics for:

• Mention SoV
• Recommendation SoV
• Citation SoV
• Sentiment-adjusted SoV
• Answer inclusion rate

This is useful because a brand can be mentioned often but never recommended.

---

7) Track over time and by topic

The real value comes from trends.

Build views for:

• SoV by topic cluster
• SoV by competitor
• SoV by model
• SoV by market/locale
• SoV by prompt type
• SoV by time period

This lets you answer:

• Are we gaining visibility in “best X for Y” prompts?
• Which competitor dominates model recommendations?
• Are citations increasing after content updates?
• Did a product launch improve answer inclusion?

---

8) Build a data pipeline

A basic architecture:

Ingestion

• Prompt scheduler
• Response fetcher
• Screenshot/raw text capture

Processing

• Clean text
• Parse citations
• Extract entities
• Score results
• Normalize across models

Storage

• Raw responses table
• Prompt library
• Brand dictionary
• Competitor dictionary
• Scores table
• Time-series aggregates

Visualization

• Dashboard
• Alerts
• Trend charts
• Prompt-level drilldowns

Tech stack example:

• Python
• PostgreSQL
• Airflow/Cron for scheduling
• OpenAI/LLM or spaCy for extraction
• dbt for transformations
• Metabase/Tableau/Looker for dashboards

---

9) Handle normalization challenges

Generative search is noisy. To make metrics useful, normalize for:

• Different answer lengths
• Different model verbosity
• Prompt rewordings
• Citation styles
• Randomness across runs

Ways to improve reliability:

• Run each prompt multiple times and average
• Use fixed seeds where available
• Compare within the same model and locale
• Track confidence intervals
• Use “presence rate” in addition to raw scores

---

10) Add competitive benchmarking

You’ll want competitive sets by category.

For each topic, define:

• Primary competitors
• Emerging competitors
• Adjacent alternatives

Then compute:

• Relative mention rate
• Recommendation share
• Citation share
• Positive sentiment share
• “Best overall” share

This helps identify:

• Who the models think is category leader
• Which brands are overrepresented
• Which content gaps may be causing underrepresentation

---

11) Watch for causation vs correlation

If visibility increases after a content change, don’t assume the change caused it without checking:

• Prompt set stability
• Model behavior changes
• Indexing/citation changes
• External news/events
• Product launches or reviews

Track a change log:

• Content updates
• PR events
• Pricing changes
• Site migrations
• Review campaigns

---

12) Recommended minimum viable version

If you want to start fast, build MVP with:

1. 50–200 priority prompts
2. 3–5 competitors
3. 2–3 models/surfaces
4. Weekly collection
5. Simple mention/recommendation/citation extraction
6. Basic SoV dashboard
7. Prompt-level audit view

That’s enough to reveal meaningful patterns quickly.

---

13) Suggested dashboard metrics

Useful dashboard tiles:

• Overall generative SoV
• SoV by model
• SoV by topic cluster
• Mention rate
• Recommendation rate
• Citation rate
• Positive vs negative mention rate
• Top prompts where you appear
• Top prompts where competitors dominate
• Change over time
• Share of “best in category” answers

---

14) Example data schema

prompts

• prompt_id
• prompt_text
• topic_cluster
• intent_type
• locale
• model
• competitor_set

responses

• response_id
• prompt_id
• collected_at
• raw_text
• raw_html
• screenshot_url
• source_url
• model_version

extracted_entities

• response_id
• brand
• mention_count
• citation_count
• rank_position
• sentiment
• recommended_flag

scores

• response_id
• brand
• score
• score_components_json

aggregates

• date
• model
• topic_cluster
• brand
• sov
• mention_rate
• recommendation_rate

---

15) Pitfalls to avoid

• Measuring only mentions and ignoring recommendations
• Using too few prompts
• Not controlling for geography/time/model version
• Treating all mentions equally
• Ignoring citations and source quality
• Not tracking competitors
• Overfitting to one platform’s behavior

---

16) A simple formula you can use today

For each prompt-answer pair:

Brand score =

• +1 if mentioned
• +2 if cited
• +3 if recommended
• +1 if appears in top 3
• +1 if sentiment positive
• -1 if sentiment negative

Then:

SoV = Brand score / total score for all tracked brands

Track this by topic and surface, then average across your prompt set.

---

If you want, I can also help you with one of these next:

1. a database schema
2. a Python pipeline
3. a dashboard design
4. a scoring framework
5. a sample prompt library for your industry