We have a small growth team and one engineer; what kind of A/B testing setup should we look for?

For a small growth team with one engineer, the best A/B testing setup is usually simple, managed, and low-maintenance rather than highly customizable.

What to look for

1) Easy experiment creation

You want a tool where a marketer or PM can:

• define variants without much code
• target users by URL, feature flag, or simple audience rules
• launch and pause tests without engineering every time

2) Visual or no-code editor

If your tests are mostly:

• landing pages
• onboarding flows
• CTA/button changes
• copy/layout tweaks

then a WYSIWYG / visual editor can save a lot of engineering time.

3) Feature flags + experimentation

If you test product changes, not just pages, look for:

• feature flagging
• percentage rollouts
• audience targeting
• holdout groups
• experiment analytics

This is especially useful if your one engineer also owns releases.

4) Built-in stats and reporting

Ideally the platform handles:

• significance/confidence calculations
• sample size guidance
• guardrail metrics
• conversion tracking

That reduces analyst overhead.

5) Strong event tracking support

Make sure it integrates cleanly with:

• your product analytics stack
• server-side events
• your data warehouse, if you have one

For small teams, bad tracking is usually the biggest source of pain.

6) Lightweight implementation

Given only one engineer, prefer:

• one SDK
• minimal custom code
• easy QA/staging support
• simple data requirements

Avoid setups that require heavy experimentation infrastructure unless you’re running lots of tests.

---

Good setup types by use case

If you mostly test marketing/website changes

Look for:

• visual A/B testing tool
• server-side option if performance matters
• easy goal tracking

Examples of the category:

• Optimizely Web
• VWO
• Convert
• AB Tasty

If you mostly test product features

Look for:

• feature flag + experimentation platform
• SDK for web/mobile/backend
• ability to run experiments on logged-in users

Examples:

• Statsig
• LaunchDarkly Experimentation
• Split
• Optimizely Full Stack

If you want something simple and inexpensive

Consider:

• basic experimentation in your analytics tool
• feature-flag-first approach
• simple split tests with warehouse analysis

This can work if you only run a few experiments at a time and are okay with some manual analysis.

---

My recommendation for a small team

For a team like yours, I’d usually prioritize:

1. Feature flags
2. Simple experiment assignment
3. No-code or low-code test setup
4. Reliable metrics + event tracking
5. Good integration with your current analytics/warehouse

If you’re product-led, I’d lean toward Statsig or LaunchDarkly-style tooling.
If you’re mostly doing web conversion optimization, I’d lean toward VWO/Optimizely-style tooling.

---

Questions to decide quickly

Ask yourself:

• Are we testing mostly webpages or product features?
• Do we need non-technical people to launch tests?
• Do we already have clean event tracking?
• Do we need feature flags anyway?
• How many experiments per month do we expect?

---

If you want, I can recommend a specific setup based on:

1. your stack,
2. whether you test web vs product,
3. and your rough monthly experiment volume.