Sample sizeTest durationBayesian probabilityBenchmarks through 13 Nov 2026

A/B test calculator: sample size, duration and Bayesian results.

Plan how many visitors each version needs and how many weeks your test should run, then check finished results with Bayesian probability. Built on the same rules Stellar uses, with benchmarks from 450 real tests.

Book a Demo
Free A/B test calculator Uses the same rules as Stellar Autopilot
Chance of detecting the lift if it's real.

How the sample size and duration calculator works

  1. Target rate: p2 = p1 × (1 + lift).
  2. Visitors per version: n = (z₁₋α/2 · √(2·p̄·(1−p̄)) + z₁₋β · √(p1(1−p1) + p2(1−p2)))² ÷ (p2 − p1)², where p̄ = (p1 + p2) ÷ 2.
  3. With more than two versions, confidence is split across the comparisons with the control (α ÷ (versions − 1)), so each comparison stays strict.
  4. We never go below 300 visitors per version.
  5. Duration = visitors per version × versions ÷ (daily visitors × share in test), rounded up to whole weeks, with a minimum of 7 days.

Example: at a 3% baseline, detecting a +20% lift at 95% confidence and 80% power takes 13,914 visitors per version. At 1,000 visitors a day split across two versions, that's 28 days.

Planning uses this classic formula. Results in Stellar, and in the Analyze tab, are reported as a Bayesian probability of beating the control. The plan tells you how much data to collect. The probability tells you what the data says.

What real tests on Stellar looked like

25th percentileMedian75th percentile
Visitors per version8272,0165,388
Runtime (days)2957116
Visitors per version per day17.439.585.1
Best variant vs control (lift)−8.1%+1.0%+13.0%

A variant beat the control with at least 95% probability in 16.9% of tests with a flagged control (445 tests, our recalculation, not in-app declared winners). Most real lifts are small, so plan for the lift you can realistically get, not the one you hope for. See the full A/B testing benchmarks or how long to run an A/B test.

Data through 13 Nov 2026 · window 28 Feb 2025 – 13 Nov 2026.

How to read Bayesian results

A direct answer.

Bayesian results tell you the probability that B beats A, for example "92% likely to be better", instead of a p-value.

Still plan your sample.

Probability is easier to read, but a small sample can still swing. Decide your sample size and duration first, then read the probability.

Pick a threshold before you start.

95% is a common bar, and it's the one Stellar Autopilot uses. Choosing it up front keeps you from moving the goalposts mid-test.

Weigh the risk.

For a cheap, easily reversed change you might act at a lower probability. For pricing or checkout changes, wait for 95% and your full sample.

Why plan with Stellar

Bayesian reporting built in.

Every Stellar test shows each version's probability of beating the control.

Autopilot with guardrails.

Autopilot keeps a winner only at 95% probability, after 7+ days and 300+ visitors per version. See Autopilot

No-code and split URL tests.

Change a page with the visual editor, or test two separate URLs.

Heatmaps on Pro and up.

See how visitors interact with each version. Use Hotjar or Clarity alongside

Pricing: Sandbox is $0 for up to 5k monthly tracked users. Plus is $29/mo billed annually for up to 25k, and Pro is $99/mo billed annually for up to 50k.