Plan how many visitors each version needs and how many weeks your test should run, then check finished results with Bayesian probability. Built on the same rules Stellar uses, with benchmarks from 450 real tests.
Example: at a 3% baseline, detecting a +20% lift at 95% confidence and 80% power takes 13,914 visitors per version. At 1,000 visitors a day split across two versions, that's 28 days.
Planning uses this classic formula. Results in Stellar, and in the Analyze tab, are reported as a Bayesian probability of beating the control. The plan tells you how much data to collect. The probability tells you what the data says.
| 25th percentile | Median | 75th percentile | |
|---|---|---|---|
| Visitors per version | 827 | 2,016 | 5,388 |
| Runtime (days) | 29 | 57 | 116 |
| Visitors per version per day | 17.4 | 39.5 | 85.1 |
| Best variant vs control (lift) | −8.1% | +1.0% | +13.0% |
A variant beat the control with at least 95% probability in 16.9% of tests with a flagged control (445 tests, our recalculation, not in-app declared winners). Most real lifts are small, so plan for the lift you can realistically get, not the one you hope for. See the full A/B testing benchmarks or how long to run an A/B test.
Data through 13 Nov 2026 · window 28 Feb 2025 – 13 Nov 2026.
Bayesian results tell you the probability that B beats A, for example "92% likely to be better", instead of a p-value.
Probability is easier to read, but a small sample can still swing. Decide your sample size and duration first, then read the probability.
95% is a common bar, and it's the one Stellar Autopilot uses. Choosing it up front keeps you from moving the goalposts mid-test.
For a cheap, easily reversed change you might act at a lower probability. For pricing or checkout changes, wait for 95% and your full sample.
Every Stellar test shows each version's probability of beating the control.
Autopilot keeps a winner only at 95% probability, after 7+ days and 300+ visitors per version. See Autopilot
Change a page with the visual editor, or test two separate URLs.
See how visitors interact with each version. Use Hotjar or Clarity alongside
Pricing: Sandbox is $0 for up to 5k monthly tracked users. Plus is $29/mo billed annually for up to 25k, and Pro is $99/mo billed annually for up to 50k.