How do I debug a feature flag rollout that only works in staging?

When a feature flag rollout works in staging but not in production, the goal is to figure out what differs between the two environments and whether the flag evaluation is actually happening the same way.

Here’s a practical debugging checklist:

1) Confirm the flag value in both environments

Check:

• Flag name/key is identical
• Percentage rollout is the same
• Targeting rules are the same
• The intended user/account is actually in the rollout segment

Also verify:

• Is the flag enabled in prod?
• Is prod using a different environment/SDK key/config than staging?

2) Compare flag evaluation context

Feature flags often depend on context like:

• user ID
• account/org ID
• email/domain
• role/plan/tier
• locale/region
• device/platform

Make sure the production request includes the same context that staging does.

Common issue:

• staging uses a test user with beta=true
• prod users don’t have the required attribute, so they never match the rule

3) Check initialization and caching

In production, flags may fail due to:

• SDK not initialized before app renders
• stale cache
• background refresh failing
• server-side vs client-side mismatch

Look for:

• delayed flag fetch
• cached flag data from previous release
• TTL not expiring
• edge/CDN caching old config

4) Inspect environment-specific config

Compare staging vs prod:

• API endpoints
• environment variables
• flag provider keys
• auth credentials
• network allowlists
• DNS/proxy settings

A very common cause is that production points to the wrong config source or uses a different project/environment in the flag service.

5) Look at logs/telemetry for flag decisions

If your flag system supports it, inspect:

• evaluation logs
• exposure events
• debug traces
• reasons for “off” decisions

You want to know:

• Was the flag evaluated?
• What rules matched?
• What attribute caused a miss?
• Was the fetch successful?

6) Reproduce with a known production-like user

Try a controlled test:

• Use the same prod user/account in staging if possible
• Or create a prod test account that matches the rollout criteria
• Compare the exact evaluation output in both envs

If staging works but prod doesn’t for the same user, that points to config or integration differences.

7) Verify release/version differences

Sometimes staging is on a newer build than production:

• flag code exists in staging but not prod
• rollout target is fine, but prod app version doesn’t contain the feature
• backend contract changed

Check:

• deployed version hash
• frontend/backend version
• migration status
• API compatibility

8) Confirm auth and permissions

Production users may fail because:

• access control blocks feature use
• backend permission checks override the flag
• feature is gated by entitlement/plan

A flag might be “on,” but the feature still won’t appear if the user lacks permission.

9) Check for country/legal/environment constraints

Sometimes production-specific logic disables features:

• region restrictions
• GDPR/consent gating
• internal IP only
• enterprise tenant rules

These often don’t exist in staging, so the rollout looks broken only in prod.

10) Use a minimal test case

Create a simple experiment:

• one user
• one flag
• one explicit allow rule
• no percentage rollout
• no secondary conditions

If that works in prod, the problem is likely in the more complex targeting or rollout logic.

---

Fastest way to isolate it

If I had to narrow it down quickly, I’d check these in order:

1. Is prod using the right flag environment/key?
2. Does the prod user match the targeting attributes?
3. Is the flag actually fetched/evaluated successfully in prod?
4. Is prod on the same app version as staging?
5. Are there permission or region overrides?

---

If you want a systematic approach

A good debugging pattern is:

• Diff the config between staging and prod
• Log the evaluation context
• Trace the flag decision
• Test one known user/account
• Verify app version and cache behavior

If you want, I can also give you:

• a step-by-step incident runbook
• a checklist for LaunchDarkly/Unleash/Statsig/etc.
• or help you debug a specific flag setup if you share the targeting rules and how it’s evaluated.