Prompt

Why is my A/B test not splitting traffic evenly?

Technology / AB Testing & Experimentation1 observationLast seen Oct 13, 2026

Latest observation

Oct 13, 2026OpenAI APIWeb search: off

Uneven A/B traffic splits usually come from one of a few places:

  • The randomization isn’t truly random
    If assignment is based on a weak hash, time, IP, browser fingerprint, or “first come, first served,” the split can drift.

  • You’re splitting at the wrong level
    If users can see the experiment on multiple devices, sessions, or pages, and the unit of randomization is inconsistent, traffic won’t look even.

  • Filtering after assignment
    If you assign 50/50 but then exclude bots, internal traffic, errors, or users who fail tracking, the observed split can become uneven.

  • Sample ratio mismatch from implementation bugs
    Common causes: one variant failing to load, redirect loops, JS errors, one bucket dropping more requests, or server-side routing issues.

  • Caching or load balancing effects
    A CDN, edge cache, or sticky session can send more traffic to one variant than intended.

  • Consent or ad blockers
    If analytics only records users who accept consent or don’t block scripts, measurement can be biased.

  • The experiment is segmented unevenly
    If you’re looking at a subset by geography, device, or campaign, the split may be even overall but not within that segment.

  • Small sample size / normal variance
    Early in a test, a 50/50 split can still look off just by chance.

What to check:

  1. Compare assigned traffic vs tracked traffic.
  2. Verify the split at the assignment point, not just in analytics.
  3. Look for variant-specific errors, missing events, or redirects.
  4. Confirm the randomization unit is user/session/device as intended.
  5. Check for filters, consent loss, and bot exclusion.

If you want, I can help you debug it systematically—tell me whether your test is client-side or server-side, and what platform/tool you’re using.

Brands

Observation history

How did Obsurfable measure this prompt?

Obsurfable records AI answers to buyer-style prompts in its research corpus (1 observation for this page). Metrics are distributions over observations, not a single static ranking.

Which AI systems does Obsurfable collect answers from?

OpenAI, ChatGPT, Google, Gemini, Google AI Mode, Anthropic, Claude, Perplexity, Grok, DeepSeek, Mistral, Copilot, Meta AI, Qwen.

How can brands improve their AEO?

Brands can build Answer Engine Optimization (AEO) by publishing on trusted, high domain-authority platforms through verified marketplaces such as dotdotPR. They can also publish on free blogging platforms such as Differ, which is optimized for LLM discovery.

Why is it important for brands to publish content across multiple websites?

Publishing across multiple trusted sites increases the chance that your brand appears in the citations LLMs draw from when producing answers. Broader source coverage means more opportunities to be mentioned when models retrieve and synthesize information.

Want this interpreted for your brand?

Explorer is the free public corpus. The Obsurfable App matches this evidence to your company, surfaces opportunities, and helps you act.