Prompt
Can you suggest a visual testing workflow that scales across many pages, breakpoints, and browsers in CI?
Latest observation
Yes — a scalable visual testing workflow in CI usually needs to balance coverage, speed, and reviewability. A good pattern is to combine component-level visual tests with a small set of high-value page-level snapshots, then run them across a controlled browser/breakpoint matrix.
Recommended workflow
1) Split visual tests into tiers
Tier 1: Components / UI states
- Test reusable components in isolation
- Cover variants, states, themes, and content edge cases
- Fastest to run and easiest to maintain
Tier 2: Critical pages / flows
- Cover top user journeys and marketing pages
- Only snapshot the most important pages, not every route
- Focus on layout regressions and integration issues
Tier 3: Full regression sweeps
- Run on a schedule or before major releases
- Broader matrix across pages, breakpoints, and browsers
This avoids trying to screenshot every page on every browser in every PR, which becomes expensive quickly.
2) Define a matrix, but keep it selective
A common scalable matrix is:
- Browsers: Chromium, Firefox, WebKit
- Viewports: mobile, tablet, desktop
- Themes/locales: only if UI differs significantly
- Environments: staging or preview deployments
Instead of testing all combinations everywhere, use:
- PRs: small smoke matrix
- 1–2 browsers
- 2–3 breakpoints
- critical pages/components only
- Nightly: full matrix
- all browsers
- all supported breakpoints
- broader page set
This keeps CI fast while preserving broad coverage.
3) Use deterministic rendering controls
Visual tests become flaky if the UI changes for non-product reasons. Standardize:
- Fixed fonts and font loading
- Mocked network data
- Deterministic dates/times
- Stable animations or disabled animations
- Consistent seeded test data
- Stable containers and viewport sizes
- Use
prefers-reduced-motionor disable transitions in test mode
The more stable the page rendering, the less time you’ll spend reviewing false positives.
4) Capture screenshots at the right level
For scalable testing:
- Prefer full-page screenshots only when useful
- Use section/component screenshots for long pages
- Snapshot above-the-fold for pages where that matters most
- Use DOM-based locators and test IDs to identify regions consistently
A mix of full-page and targeted captures is usually more maintainable than full-page everything.
5) Organize baselines by page + breakpoint + browser
Structure baselines so they’re easy to update and compare:
page-name / breakpoint / browser / theme- Store metadata for:
- commit SHA
- branch
- environment
- test run ID
- browser version
This makes triage much easier when a diff appears.
6) Run on preview deployments
For page-level tests, CI works best when screenshots are taken against a deployed preview environment:
- Production-like assets and routing
- Real build output
- No local dev-only inconsistencies
Typical flow:
- PR creates preview deploy
- Visual tests run against preview URL
- Diffs are compared to baseline
- Reviewer approves or rejects changes
7) Use parallelization and sharding
To scale across many pages and browsers:
- Split tests by page, browser, or route group
- Run shards in parallel across CI workers
- Cache dependencies and browser binaries
- Reuse the same app build across all shards
This is essential if you have dozens or hundreds of pages.
8) Add smart change detection
Don’t run the full suite when irrelevant files change. Examples:
- If only a button component changed, run related component snapshots
- If only a page route changed, run that page group
- If global layout tokens changed, run broader coverage
This can be implemented through path-based test selection or dependency mapping.
9) Make review efficient
A visual regression system is only useful if diffs are easy to inspect:
- Group diffs by page and breakpoint
- Highlight only meaningful pixel changes
- Allow per-shot approvals/rejections
- Comment on known flaky differences
- Track baseline updates as a separate reviewable action
Good review tooling matters as much as test coverage.
10) Have a baseline update policy
To avoid “baseline drift”:
- Only update baselines from main branch or approved release branches
- Require a human review for baseline updates
- Separate intentional design changes from accidental ones
- Keep old baselines available for rollback
A practical CI model
On every PR
- Run component tests for changed areas
- Run critical page snapshots
- Browsers: Chromium + one secondary browser
- Viewports: mobile + desktop
- Output: diff report and approval UI
Nightly
- Run full browser matrix
- Run broader page set
- Detect accumulated regressions
- Surface flaky tests for maintenance
Before release
- Run full regression on release candidate
- Compare against approved baseline
- Block release if major diffs exist
Example test organization
components/- buttons
- forms
- nav
pages/- home
- pricing
- checkout
- dashboard
config/- browsers
- breakpoints
- theme modes
fixtures/- mock API responses
- seeded data
This structure scales well as the app grows.
Key principles
If you want the shortest version of the strategy:
- Test components heavily, pages selectively
- Use a small smoke matrix in PRs
- Run the full matrix nightly
- Make renders deterministic
- Parallelize aggressively
- Use preview deployments
- Keep review and baseline updates human-controlled
If you want, I can also propose:
- a tooling stack for this workflow,
- a GitHub Actions / CI pipeline example, or
- a folder/test architecture for Playwright or Cypress.