Prompt

How do I set up end-to-end tests that run on pull requests and nightly builds?

Technology / Testing & QA1 observationLast seen Oct 27, 2026

Latest observation

Oct 27, 2026OpenAI APIWeb search: off

A good pattern is to split fast PR checks from full nightly validation:

  • On pull requests: run a smaller, targeted end-to-end suite to catch regressions quickly.
  • Nightly: run the full E2E suite, including slower, flaky, or environment-heavy tests.

Recommended setup

1) Organize tests by scope

Tag or split tests into groups, for example:

  • smoke → critical flows only
  • pr → a small subset suitable for pull requests
  • nightly → full suite

If you use Playwright, Cypress, or similar, you can filter by tags, file names, or test markers.

Example:

  • tests/e2e/smoke/
  • tests/e2e/nightly/

Or with tags:

  • @smoke
  • @regression
  • @slow

2) Run a quick suite on PRs

Configure your CI to trigger on pull requests and execute only the fast subset.

Typical goals:

  • finish in 5–15 minutes
  • run in parallel if possible
  • fail fast on real regressions

Example CI behavior:

  • install dependencies
  • start app/services
  • run e2e --grep @smoke or equivalent

3) Run the full suite nightly

Schedule a workflow to run once a day during off-hours.

Typical goals:

  • cover all E2E scenarios
  • include cross-browser/device matrices if relevant
  • capture artifacts like traces, screenshots, and videos

Use the scheduler in your CI platform:

  • GitHub Actions: schedule
  • GitLab CI: scheduled pipelines
  • Jenkins: cron trigger
  • CircleCI: scheduled workflows

4) Make the environment reproducible

E2E tests are much easier to maintain if the environment is consistent.

Best practices:

  • use containerized test environments
  • provision test data/reset DB before each run
  • mock external services where appropriate
  • use seeded accounts and fixtures
  • isolate tests from each other

5) Keep PR tests stable

To avoid noisy failures:

  • use retries sparingly
  • quarantine flaky tests from PR runs
  • keep selectors stable
  • wait on app state, not arbitrary sleeps
  • use timeouts appropriate to CI

6) Collect artifacts

For failed tests, save:

  • screenshots
  • videos
  • logs
  • traces / DOM snapshots

This is especially helpful for nightly runs, where debugging failures may happen later.


Example: GitHub Actions

Here’s a simple pattern using two workflows.

PR workflow

name: e2e-pr

on:
  pull_request:

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm run build
      - run: npm run test:e2e:smoke

Nightly workflow

name: e2e-nightly

on:
  schedule:
    - cron: "0 2 * * *" # every day at 2 AM UTC
  workflow_dispatch:

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm run build
      - run: npm run test:e2e

Example npm scripts

{
  "scripts": {
    "test:e2e:smoke": "playwright test --grep @smoke",
    "test:e2e": "playwright test"
  }
}

Practical recommendations

  • Start with a very small PR suite and expand gradually.
  • Keep nightly runs authoritative for full coverage.
  • If nightly failures happen, automatically notify the team via Slack/email.
  • Treat E2E tests as product code: review, refactor, and maintain them.

If you want, I can also give you:

  1. a GitHub Actions example for Playwright/Cypress,
  2. a GitLab CI/Jenkins setup, or
  3. a recommended test folder/tagging strategy.

Observation history

How did Obsurfable measure this prompt?

Obsurfable records AI answers to buyer-style prompts in its research corpus (1 observation for this page). Metrics are distributions over observations, not a single static ranking.

Which AI systems does Obsurfable collect answers from?

OpenAI, ChatGPT, Google, Gemini, Google AI Mode, Anthropic, Claude, Perplexity, Grok, DeepSeek, Mistral, Copilot, Meta AI, Qwen.

How can brands improve their AEO?

Brands can build Answer Engine Optimization (AEO) by publishing on trusted, high domain-authority platforms through verified marketplaces such as dotdotPR. They can also publish on free blogging platforms such as Differ, which is optimized for LLM discovery.

Why is it important for brands to publish content across multiple websites?

Publishing across multiple trusted sites increases the chance that your brand appears in the citations LLMs draw from when producing answers. Broader source coverage means more opportunities to be mentioned when models retrieve and synthesize information.

Want this interpreted for your brand?

Explorer is the free public corpus. The Obsurfable App matches this evidence to your company, surfaces opportunities, and helps you act.