A/B testing

A/B testing software built on feature flags

Every partial rollout already splits your traffic. Toggled turns that split into an experiment: attach a metric, keep sticky cohorts, and know whether the feature helped before it reaches everyone. No separate experimentation contract.

No card required

01 · The mechanism

A rollout with a hypothesis attached

An experiment in Toggled is a flag with variants and a metric. The same consistent hashing that powers a gradual rollout assigns each user a stable variant, so nobody sees both versions and your data stays clean across sessions and services.

Primary metric: the one number the feature exists to move, like checkout conversion.
Guardrails: error rate, latency, refunds. A win that breaks reliability is not a win.
Sticky cohorts: assignment by user id, not by request, in every SDK the same way.

experiment: pricing-page.v2

Illustrative sample data

A · current page · 50% 8.4%

B · new page · 50% 10.1%

Confidence

96%

Error rate

flat

p95 latency

+1 ms

02 · Why build tests on flags

One system for shipping and learning

No second integration

The flag SDK you ship with is the experiment SDK. No extra snippet slowing your page, no second user-identity mapping to keep in sync.

From result to rollout in one motion

When B wins, you do not export a report and file a ticket. You drag the same flag to 100% and delete the loser path.

Experiments where your code runs

Test backend algorithms, API changes and pricing logic, not just button colors. If a flag can gate it, a metric can measure it.

03 · Reading results honestly

The tool will not fool you, so do not fool yourself

Most A/B testing mistakes are human, not statistical. The console is designed to make the honest reading the easy one:

Declare the primary metric before you start; the readout keeps it front and center.
Peeking is handled: sequential analysis on Scale plans keeps early looks from inflating false wins.
Segment breakdowns are labeled exploratory, so a fished-up "win for Android users in Canada" reads as a hypothesis, not a result.

When not to test

Sometimes the answer is just ship it

A/B testing has a cost: time at partial rollout and attention on dashboards. Skip it when the decision does not need it:

Bug fixes and compliance work: correctness is not a preference question.
Tiny traffic: below a few thousand users per variant, most product changes cannot reach significance in reasonable time. Use a canary deployment for safety and ship.
One-way doors: if you cannot keep the old path alive, a test cannot save you from the decision.

More depth in the post on A/B testing with feature flags.

Early access · launching soon

Measure the next feature before you commit

Experimentation is included in every Toggled plan, not sold as a second product. Join the early-access list; no card required.

No card required · your data stays yours

Keep exploring