Prompt
How do I compare screenshots automatically in CI?
Latest observation
Oct 27, 2026
OpenAI APIWeb search: off
A common way is to use visual regression testing: generate screenshots in CI, compare them against approved “baseline” images, and fail the build if the diff is above a threshold.
Typical approaches
1) Use a screenshot-diff tool
Popular options:
- Playwright: built-in
toMatchSnapshot()/ screenshot assertions - Puppeteer +
jest-image-snapshot - Cypress + screenshot comparison plugins
- Standalone tools like BackstopJS, Percy, Chromatic (for Storybook)
Recommended setup with Playwright
If you already use Playwright, this is usually the simplest.
Example test
import { test, expect } from '@playwright/test';
test('homepage visual', async ({ page }) => {
await page.goto('https://example.com');
await expect(page).toHaveScreenshot('homepage.png');
});
How it works
- First run: creates baseline screenshot
- Later runs: compares current screenshot to baseline
- If there are differences, the test fails
Handling CI stability
Screenshots are often flaky unless you control the environment.
Make rendering deterministic
- Use the same browser version in CI
- Fix viewport size
- Disable animations
- Use consistent fonts
- Mock dates, random values, and network data
- Wait for the page to settle before capturing
Playwright config example
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
viewport: { width: 1280, height: 720 },
locale: 'en-US',
timezoneId: 'UTC',
screenshot: 'only-on-failure',
},
expect: {
toHaveScreenshot: {
maxDiffPixels: 100,
threshold: 0.2,
},
},
});
CI pipeline flow
- Build app
- Start app or deploy preview
- Run screenshot tests
- Upload diff artifacts on failure
- Fail the job if diffs exceed threshold
Useful comparison methods
- Pixel-by-pixel diff: strict, but sensitive to small rendering changes
- Perceptual diff: more tolerant to tiny anti-aliasing differences
- Threshold-based diff: allows small, acceptable changes
Best practices
- Keep baselines in version control or in a visual testing service
- Review diffs before updating baselines
- Separate stable pages from highly dynamic ones
- Compare only the relevant region if possible
- Use a small diff threshold, not zero, to avoid flaky failures
If you want a quick solution
- Playwright if you want open-source and integrated testing
- Percy/Chromatic if you want hosted visual review and approvals
If you want, I can give you:
- a Playwright CI example,
- a Cypress example, or
- a GitHub Actions workflow for screenshot comparison.