Prompt
How do I set up visual regression testing in CI for a frontend app?
Latest observation
A solid visual regression setup in CI usually has 5 parts:
- Render pages/components in a controlled browser
- Capture screenshots
- Compare them against a baseline
- Fail CI on unexpected diffs
- Provide a way to review/update baselines intentionally
Here’s a practical approach.
1) Pick a tool
Common options:
- Playwright — great for modern apps, easy screenshot testing, good CI support
- Cypress + screenshot diff plugin — workable, but Playwright is often simpler for visual testing
- Chromatic — excellent if you use Storybook
- Percy — hosted visual review service, integrates with many frameworks
If you want a fully self-managed setup, Playwright is a strong default.
2) Define what you want to test
Visual regression is best for:
- key pages
- important UI states
- component stories
- responsive breakpoints
- dark/light mode
- error/loading/empty states
Avoid testing:
- highly dynamic content
- ads, timestamps, randomized data
- third-party widgets unless mocked
3) Create stable screenshots
To reduce flaky diffs, make screenshots deterministic:
- use a fixed viewport size
- set consistent locale/timezone
- mock network requests
- use seeded/fixed test data
- disable animations
- wait for fonts to load
- hide unstable elements like cursors, carousels, maps, or live widgets
Example Playwright settings:
viewport: { width: 1280, height: 720 }timezoneId: 'UTC'locale: 'en-US'
4) Write screenshot tests
Example with Playwright
Install:
npm i -D @playwright/test
npx playwright install
Example test:
import { test, expect } from '@playwright/test';
test('home page visual', async ({ page }) => {
await page.goto('http://localhost:3000');
await expect(page).toHaveScreenshot('home.png', {
fullPage: true,
});
});
If you want to test a specific element:
await expect(page.locator('[data-testid="hero"]')).toHaveScreenshot('hero.png');
5) Run it in CI
In CI, you usually:
- install dependencies
- start the app
- run visual tests
- upload artifacts if tests fail
Example GitHub Actions workflow:
name: visual-regression
on:
pull_request:
push:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps
- name: Build app
run: npm run build
- name: Start app
run: npm run start &
- name: Wait for app
run: npx wait-on http://localhost:3000
- name: Run visual tests
run: npx playwright test
If your app needs a preview server, use that instead of start.
6) Store baselines properly
When screenshots are first created, they become your baseline.
Typical workflow:
- PR runs compare current screenshots vs baseline
- if there are intentional UI changes, update baselines in a separate commit/PR
- review diffs before accepting changes
With Playwright, update baselines locally:
npx playwright test --update-snapshots
Then commit the updated screenshots.
7) Review diffs clearly
Good visual regression tools should show:
- current screenshot
- baseline screenshot
- diff overlay
If using Playwright alone, failed test artifacts can be attached from CI. If you want a better review UI, consider:
- Chromatic
- Percy
These are especially nice when lots of screenshots fail and you want a human-friendly review flow.
8) Handle common flake sources
Visual tests often fail due to tiny non-functional changes. Common fixes:
- set
font-smoothingor wait for fonts to load - replace animated GIFs/video with static mocks
- freeze date/time
- mock random IDs
- disable CSS transitions/animations in test mode
- avoid relying on OS-specific rendering differences
- use the same browser in CI and local runs
You can also slightly relax diffs:
- pixel threshold
- maximum diff percentage
- ignore dynamic regions
Example:
await expect(page).toHaveScreenshot('home.png', {
fullPage: true,
threshold: 0.2,
});
9) Suggested practical setup
For most frontend teams:
If you already use Storybook
- Use Chromatic
- Add visual tests per component story
- Great PR review experience
If you test full pages and flows
- Use Playwright
- Snapshot important pages at key breakpoints
- Run in GitHub Actions/GitLab CI
10) Recommended CI strategy
A good starting policy:
- run visual tests on every PR
- fail the PR if diffs exceed threshold
- allow baseline updates only on approved branches or via maintainer action
- keep visual snapshots versioned in git, or use hosted baseline storage
Minimal checklist
- choose Playwright/Chromatic/Percy
- fix viewport and environment
- mock data and time
- capture screenshots for key states
- compare in CI
- review diffs before updating baselines
- add artifact upload on failure
If you tell me your stack, I can give you a ready-to-use setup:
- React / Next.js / Vue / Angular
- Storybook or not
- GitHub Actions / GitLab CI / CircleCI
- Playwright / Cypress / Chromatic / Percy