Prompt
I'm building a release process where feature flags and experiments live together
Latest observation
That’s a strong pattern. A good way to think about it is:
- Feature flags control whether something is available.
- Experiments control how you measure and compare behavior.
If you want them to “live together,” the key is to avoid treating them as separate systems with separate lifecycle rules. Instead, make them part of one release/decision framework.
A practical model
1. Use one flag object, two modes
Every launch should have a single configurable entity that can act as:
- Release flag: on/off, targeted rollout, kill switch
- Experiment flag: variant assignment, traffic split, metrics attachment
Example fields:
keytype:release | experimentstate:draft | active | archivedvariants:control,treatment-a, etc.allocation: percentage splittargeting: audience rulesownerstart_date,end_datesuccess_metric,guardrail_metrics
This lets you manage both feature rollout and experimentation from the same workflow.
2. Separate “delivery” from “decision”
A feature flag decides delivery. An experiment decides evidence.
A good workflow:
- Create flag
- Attach hypothesis + metrics
- Roll out to internal users
- Run experiment
- Promote winning variant
- Archive the flag
This avoids the common anti-pattern where experiment logic stays in code forever.
3. Treat experiments as temporary flags
Experiments should be time-boxed and lifecycle-managed.
Recommended states:
CreatedRunningAnalyzingPromoted/StoppedArchived
And enforce:
- owner
- expiry date
- cleanup ticket
- code removal after conclusion
This keeps the system from becoming a permanent switch graveyard.
What to design for
Identity and bucketing
Use stable assignment so a user sees the same variant across sessions.
Common keys:
- user ID
- account ID
- workspace/org ID
- device ID if necessary
Define the bucketing unit up front, especially if your experiments are in B2B, where account-level randomization is often better than user-level.
Guardrails
Don’t only measure primary metrics. Add guardrails like:
- latency
- error rate
- conversion funnel drop-off
- support tickets
- revenue impact
A feature can “win” on a click metric but still harm the product.
Targeting and eligibility
A user should only enter an experiment if they’re eligible.
Typical exclusion rules:
- employees/internal users
- existing customers vs new customers
- unsupported platforms
- edge-case accounts
- users already exposed to prior experiments
Eligibility should be evaluated before random assignment.
Conflict handling
When feature flags and experiments coexist, conflicts happen:
- user is in two experiments affecting the same surface
- a kill switch overrides treatment
- a rollout flag changes mid-experiment
You need precedence rules.
A common hierarchy:
- Safety overrides / kill switches
- Eligibility exclusions
- Experiment assignment
- Gradual rollout
- Default behavior
Document this clearly so teams know what “wins.”
Suggested architecture
Flag evaluation service
At runtime, your app asks a single service:
- what variant should this entity get?
- what flags are enabled?
- what experiment assignment applies?
The service should return:
- flag value
- variant
- reason
- rule matched
- experiment ID
This is useful for debugging and analytics.
Event logging
Log both exposure and outcome events.
Minimum:
flag_exposedvariant_assignedmetric_eventflag_exitedorexperiment_ended
Without exposure logging, your experiment analysis will be weak or misleading.
Good operational rules
- Every experiment has an owner
- Every experiment has an end date
- Every experiment has a success criterion
- Every flag has a cleanup plan
- Every launch has a kill switch
- Never leave experiment code in place indefinitely
- Avoid overlapping experiments on the same user journey unless designed for it
A simple policy you can adopt
All product changes must be launched behind a managed flag.
If the change is measurable, the flag must support experiment assignment and reporting.
If the flag is temporary, it must have an expiry date and cleanup task.
That gives you one coherent release process.
If you want, I can help you turn this into:
- a process diagram,
- a data model/schema, or
- a policy/playbook for engineering teams.