Prompt
I’m building a microservices system and need feature flags across services, how should I do it?
Latest observation
For microservices, the best pattern is usually:
1) Use a centralized feature-flag service
Have one source of truth for flags, with:
- flag definitions stored centrally
- targeting rules by service, user, tenant, environment, region, etc.
- audit/history
- role-based access
- gradual rollout support (percentage, cohort, allowlist)
Good options:
- Managed: LaunchDarkly, Split, ConfigCat, Unleash Cloud
- Self-hosted: Unleash, Flagsmith
2) Evaluate flags locally in each service
Avoid calling the flag service on every request.
Instead:
- each service caches flag state
- refreshes periodically or via streaming updates
- evaluates flags in-process using local context
This gives:
- low latency
- resilience if the flag service is unavailable
- less cross-service coupling
3) Distinguish “release flags” from “runtime config”
Use feature flags for:
- gradual releases
- experiments
- kill switches
- per-tenant enablement
Don’t use them for:
- permanent application settings
- secrets
- core business rules that should be in code/config
A good rule: if a flag will live forever, it’s probably not a feature flag.
4) Standardize context and naming
Define a shared context model across services, for example:
serviceenvironmenttenant_iduser_idregionrolerequest_attributes
And use consistent naming:
billing.new_pricing_flowsearch.enable_vector_rankingcheckout.disable_coupon_stack
5) Put flags behind a shared library or SDK
Create a common internal wrapper:
- consistent API across languages
- common defaults/fallback behavior
- logging/metrics hooks
- testing helpers
Example conceptually:
if flags.is_enabled("checkout.new_payment_flow", context):
...
6) Make failure behavior explicit
Decide what happens if flag evaluation fails:
- default to off for risky features
- default to on for safe telemetry/UX improvements
- log and metric every fallback
For critical paths, implement:
- cached last-known-good values
- timeout budgets
- circuit breaker around the flag backend
7) Handle rollout and safety operationally
Support:
- staged rollout: 1% → 10% → 50% → 100%
- kill switches for fast rollback
- automatic rollback signals from error rate/latency
- ownership and expiry dates for flags
Track:
- who created the flag
- why it exists
- when it should be removed
8) Avoid distributed inconsistency pitfalls
In microservices, two services may see different flag states temporarily. Plan for that:
- design flag-driven changes to be backward compatible
- deploy code that supports both old/new behavior during rollout
- avoid flipping a flag that requires perfect simultaneity across services
Use patterns like:
- expand/contract migrations
- dual reads/writes
- compatibility layers
9) Consider where evaluation happens
Common options:
A. Client/service-side evaluation
Best for:
- latency
- resilience
- complex targeting
B. Gateway/API layer evaluation
Best for:
- coarse routing decisions
- request shaping
C. Central evaluation per request
Usually avoid unless you truly need it; it adds latency and a hard dependency.
10) Add governance from day one
Feature flags can become technical debt quickly. Add:
- flag TTL/expiry
- cleanup tickets
- dashboards for stale flags
- ownership per team/service
- naming conventions and review process
Practical recommendation
For most teams, I’d do this:
- Choose Unleash / Flagsmith / LaunchDarkly
- Expose a shared flag SDK to all services
- Use local cached evaluation with periodic refresh/streaming
- Standardize context + naming
- Build flag cleanup and expiry into your process
- Ensure all flag-driven releases are backward compatible
A simple architecture
- Feature Flag Service
- stores flag definitions
- serves updates
- Service SDKs
- fetch updates
- cache locally
- evaluate flags in-process
- Observability
- logs flag decisions
- metrics for evaluation errors, stale cache, rollout percentage
- Governance
- ownership, TTL, approval, cleanup
If you want, I can also sketch:
- a reference architecture diagram,
- a language-agnostic SDK design, or
- a rollout/flag lifecycle process for your team.