Prompt
How do I compare two datasets on coverage, freshness, and data quality?
Latest observation
A practical way to compare two datasets is to score them across three dimensions—coverage, freshness, and data quality—using the same definitions and metrics for both. Here’s a simple framework.
1) Coverage
Coverage tells you how much of the target universe each dataset contains.
Common questions
- Does the dataset include all expected entities/records?
- How complete is it by geography, segment, category, or time period?
- Are there missing slices that matter to your use case?
Useful metrics
- Record count comparison: total rows/entities in each dataset
- Entity overlap:
A ∩ B= records present in bothA \ B= records only in AB \ A= records only in B
- Coverage rate vs a reference:
coverage = dataset_count / expected_count
- Attribute coverage: percent of non-null values for important fields
- Segment coverage: coverage by region, product, customer type, etc.
Example
If you expect 100,000 customers and Dataset A has 92,000 while Dataset B has 97,000:
- A coverage = 92%
- B coverage = 97%
If some regions are missing in A but present in B, that may matter more than the raw count.
2) Freshness
Freshness tells you how up-to-date the data is.
Common questions
- How recently was the data updated?
- How far behind real time is it?
- Does it refresh consistently?
Useful metrics
- Data latency:
current_time - latest_record_timestamp
- Ingestion lag:
load_time - source_event_time
- Update frequency: hourly, daily, weekly
- Staleness by partition: how old each day/segment is
- Refresh consistency: % of expected refreshes that arrived on time
Example
- Dataset A latest timestamp: 2 hours ago
- Dataset B latest timestamp: 1 day ago
Then A is fresher, even if B has more records.
3) Data quality
Data quality tells you how trustworthy and usable the data is.
Common dimensions
- Completeness: missing values
- Validity: values conform to rules/formats/ranges
- Accuracy: values match reality or a trusted source
- Consistency: no contradictions across fields/tables
- Uniqueness: no duplicates where there shouldn’t be
- Integrity: foreign keys / relationships are intact
Useful metrics
- Null rate by field
- Duplicate rate
- Rule violation rate
Examples:- negative quantities where impossible
- invalid email formats
- dates in the future when not allowed
- Referential integrity failures
- Outlier rate
- Error rate from audit/reconciliation checks
Example
- Dataset A has 1% missing email addresses and 0.5% duplicates
- Dataset B has 8% missing email addresses and 0.1% duplicates
A may be higher quality for contactability, while B may be better for identity uniqueness.
4) Compare them side by side
A simple scorecard works well:
| Dimension | Dataset A | Dataset B | Winner |
|---|---|---|---|
| Coverage | 92% | 97% | B |
| Freshness | 2 hours lag | 24 hours lag | A |
| Missing values in key fields | 1.2% | 4.8% | A |
| Duplicate rate | 0.5% | 0.1% | B |
| Validity rule pass rate | 98.7% | 96.1% | A |
Then decide based on your use case:
- If you need the most complete historical view: prioritize coverage
- If you need current data: prioritize freshness
- If you need reliable analytics or operations: prioritize quality
5) Use a weighted score if needed
If one factor matters more than others, assign weights.
Example:
- Coverage: 40%
- Freshness: 30%
- Quality: 30%
Normalize each metric to a 0–100 scale, then compute:
overall score = 0.4*coverage + 0.3*freshness + 0.3*quality
This makes the comparison explicit and repeatable.
6) Important best practices
- Define the same unit of comparison in both datasets
- Compare against a reference or expected baseline if possible
- Break results down by segment to avoid hiding gaps
- Check critical fields separately from non-critical ones
- Validate with a small set of ground-truth samples if accuracy matters
7) Recommended workflow
- Identify the target population or expected universe
- Compute coverage metrics
- Compute freshness metrics
- Run data quality checks
- Summarize in a scorecard
- Weight metrics based on business priority
- Review exceptions by segment and field
If you want, I can also give you:
- a SQL template for comparing two tables,
- a Python/pandas example, or
- a ready-to-use scorecard template.