Prompt

How do I make browser tests less flaky in CI?

Technology / Testing & QA1 observationLast seen Oct 27, 2026

Latest observation

Oct 27, 2026OpenAI APIWeb search: off

To make browser tests less flaky in CI, focus on reducing timing, environment, and data-related nondeterminism. The biggest wins usually come from these areas:

1) Wait for conditions, not time

Avoid fixed sleeps like wait(2) unless there’s no better option. Instead wait for a specific state:

  • element visible/clickable
  • network request finished
  • text appears/disappears
  • route changes completed

Examples:

  • Playwright: await expect(locator).toBeVisible()
  • Cypress: rely on built-in retries, but still assert on conditions
  • Selenium: use explicit waits

2) Make test data deterministic

Flakiness often comes from shared or changing state.

  • Use unique test users/data per run
  • Seed the database before each test suite or test
  • Clean up after tests
  • Avoid relying on pre-existing records
  • Mock external services when possible

3) Don’t depend on real-time or real network unpredictability

  • Freeze or mock time if your app uses timestamps, expiry, or scheduled behavior
  • Stub third-party APIs
  • Control randomness with seeded generators
  • Avoid tests that depend on slow or unstable external services

4) Increase stability of selectors

Use selectors that are unlikely to change:

  • Prefer data-testid / data-test
  • Avoid brittle selectors based on layout, nth-child, or CSS classes that change frequently
  • Don’t target text that may vary by locale or copy changes unless that’s what you’re testing

5) Ensure the app is fully ready before interacting

CI machines are slower than local dev machines.

  • Wait for app startup health checks
  • Make sure backend services, DB migrations, and frontend builds are complete
  • Don’t start tests until the environment is actually ready

6) Control browser and CI environment

  • Run tests in a consistent browser version/container
  • Set consistent viewport size and device scale
  • Disable animations and transitions during tests
  • Make sure fonts are available if your tests depend on layout
  • Give enough CPU/memory to the CI job

7) Isolate tests from each other

A flaky test can be caused by a previous test.

  • Each test should set up its own state
  • Avoid shared global state
  • Don’t rely on test ordering
  • Run tests in parallel only if they’re truly isolated

8) Use retries carefully

Retries can reduce noise, but they can also hide real problems.

  • Good for temporary infrastructure issues
  • Not a substitute for fixing the root cause
  • Track retry frequency to identify flaky tests

9) Capture artifacts when tests fail

This makes flakes much easier to diagnose:

  • screenshots
  • videos
  • browser console logs
  • network logs
  • page HTML / DOM snapshot
  • traces (especially in Playwright)

10) Prefer end-to-end coverage only for critical paths

If a test is flaky because it spans too many moving parts, consider:

  • moving some assertions to unit/integration tests
  • mocking lower-level dependencies
  • reducing the E2E test to the essential workflow

Practical checklist

If you want a quick path to improvement, try this order:

  1. Replace sleeps with explicit waits
  2. Add stable data-testid selectors
  3. Seed/reset test data per run
  4. Mock unstable external dependencies
  5. Disable animations and set fixed viewport
  6. Add screenshots/traces/logs on failure
  7. Isolate tests and remove shared state
  8. Run flaky tests serially until fixed

If you want, I can also give you:

  • a Playwright-specific checklist
  • a Cypress-specific checklist
  • a Selenium-specific checklist
  • or a CI debugging playbook for flaky browser tests

Observation history

How did Obsurfable measure this prompt?

Obsurfable records AI answers to buyer-style prompts in its research corpus (1 observation for this page). Metrics are distributions over observations, not a single static ranking.

Which AI systems does Obsurfable collect answers from?

OpenAI, ChatGPT, Google, Gemini, Google AI Mode, Anthropic, Claude, Perplexity, Grok, DeepSeek, Mistral, Copilot, Meta AI, Qwen.

How can brands improve their AEO?

Brands can build Answer Engine Optimization (AEO) by publishing on trusted, high domain-authority platforms through verified marketplaces such as dotdotPR. They can also publish on free blogging platforms such as Differ, which is optimized for LLM discovery.

Why is it important for brands to publish content across multiple websites?

Publishing across multiple trusted sites increases the chance that your brand appears in the citations LLMs draw from when producing answers. Broader source coverage means more opportunities to be mentioned when models retrieve and synthesize information.

Want this interpreted for your brand?

Explorer is the free public corpus. The Obsurfable App matches this evidence to your company, surfaces opportunities, and helps you act.