Prompt
What's the best way to do visual regression testing for a component library with lots of variants and dynamic content?
Latest observation
For a component library with lots of variants and dynamic content, the best approach is usually a layered visual regression strategy, not “take screenshots of everything.”
Recommended approach
1. Test at the component level, not the app level
Use a component explorer like:
- Storybook
- Ladle
- Histoire
Create stories for each component state:
- default
- hover/focus/disabled
- light/dark theme
- size variants
- error/loading/empty states
- responsive breakpoints
This gives you a controlled surface for screenshots and makes failures easier to debug.
2. Automate screenshots with a browser tool
Use:
- Playwright for screenshots
- or a visual testing platform like:
- Chromatic (excellent with Storybook)
- Percy
- Applitools (stronger for dynamic/AI-assisted diffs)
For most libraries, a good baseline is:
- Storybook + Chromatic, or
- Storybook + Playwright screenshot tests
3. Limit combinatorial explosion with a “state matrix”
Don’t render every possible combination. Instead:
- identify critical axes:
- theme
- size
- density
- content length
- interaction state
- locale
- then choose a small representative set for each component
Example:
- Button: 3 variants × 3 sizes × 2 themes × 4 states = too many
- Better: choose only the most meaningful combinations plus edge cases
This keeps tests maintainable.
4. Stabilize dynamic content
Dynamic content is the main source of flaky visual tests. Reduce variability by:
- seeding random data
- mocking APIs
- freezing dates/times
- using deterministic IDs
- disabling animations/transitions
- controlling fonts and loading states
- using fixed avatars/images or generated placeholders
If content is truly variable, test:
- structure and layout
- bounds and clipping
- typography wrapping rather than exact text snapshots.
5. Use masking and thresholds carefully
For dynamic regions, use:
- masked areas for timestamps, avatars, counters, ads, etc.
- diff thresholds to ignore tiny antialiasing differences
- region-based comparisons if only part of the component matters
Avoid too-permissive thresholds, or you’ll miss real regressions.
6. Prefer stable environments
Make screenshots reproducible by fixing:
- viewport size
- browser version
- OS if possible
- font rendering environment
- DPR/device scale factor
If you can, run tests in a consistent CI image.
7. Combine visual tests with non-visual assertions
Visual regression shouldn’t do everything. Pair it with:
- unit tests for logic
- accessibility tests
- interaction tests
- layout assertions when needed
This reduces the number of screenshot cases you need.
Practical “best” setup
If you want a strong default stack:
- Storybook for variant cataloging
- Chromatic for visual regression at scale
- Playwright for custom edge cases and interaction-based screenshots
- Deterministic mocks/fixtures for dynamic content
That’s often the best balance of:
- developer experience
- coverage
- maintainability
Rules of thumb
- Screenshot stories, not whole pages, unless you need integration coverage.
- Keep each story single-purpose.
- Mock all external data.
- Snapshot important states, not every permutation.
- Review diffs manually, especially early on.
- Treat visual tests as a signal, not absolute truth.
If you want, I can also suggest:
- a recommended Storybook/Chromatic setup, or
- a Playwright-based visual testing architecture for a component library.