I’m building a monitoring setup for AI Overviews, how do I store citation history?

For AI Overviews monitoring, the key is to store citation snapshots over time, not just the latest result.

What to store

For each query/run, save:

• Query metadata

  • query text
  • locale / country
  • device
  • language
  • date/time collected
  • search engine / source

• AI Overview snapshot

  • overview text/summary
  • whether an overview was present
  • position/placement if relevant
  • version hash of the overview text

• Citations / sources

  • cited URL
  • source title
  • domain
  • citation order/position
  • snippet or cited passage if available
  • source type if inferred
  • confidence/notes

• Run metadata

  • crawl/collection job ID
  • parser version
  • HTML/raw response reference if you keep it

Best storage pattern

Use a normalized model with three layers:

1) Query table

One row per monitored query.

• query_id
• query_text
• locale
• device
• created_at

2) Snapshot table

One row per collection event.

• snapshot_id
• query_id
• collected_at
• overview_present
• overview_text
• overview_hash
• raw_result_ref

3) Citation table

One row per citation in a snapshot.

• citation_id
• snapshot_id
• rank
• url
• domain
• title
• snippet
• display_url
• citation_hash

This lets you compare citations across time and detect:

• new citations
• removed citations
• rank/order changes
• text changes in the overview
• source churn by query

Important design choice: store history by snapshot, not overwrite

Do not store citations as a mutable list on the query record.
Instead, each run gets its own immutable snapshot. That preserves change history and makes diffs easy.

Helpful deduping keys

To compare citations reliably, normalize:

• canonical URL
• domain
• trailing slashes
• tracking parameters
• redirect targets

A good unique key for a citation within a snapshot is often:

• snapshot_id + canonical_url + rank

For cross-snapshot matching:

• canonical URL
• domain
• title similarity if URLs vary

Example schema

queries(
  id PK,
  query_text,
  locale,
  device,
  created_at
)

overview_snapshots(
  id PK,
  query_id FK,
  collected_at,
  overview_present,
  overview_text,
  overview_hash,
  raw_ref
)

overview_citations(
  id PK,
  snapshot_id FK,
  citation_rank,
  url,
  canonical_url,
  domain,
  title,
  snippet,
  citation_hash
)

If you want diffing

Create a derived table or job that compares consecutive snapshots:

• added citations
• dropped citations
• changed rank
• overview text changed
• citation text changed

Practical tips

• Keep the raw HTML/JSON if allowed, for parser debugging.
• Use UTC timestamps.
• Version your parser because citation extraction can change.
• Store a content hash for overview text and citation URLs/snippets.

If you want, I can also sketch:

1. a PostgreSQL schema,
2. a JSON document model,
3. or a diff algorithm for citation changes.