A/B Test Sample Size Calculator
Marketing ToolsHow much traffic a test needs before its result means anything — power included.
What the control converts at today. A lower rate needs far more traffic.
5% rising to 5.50%. Anything smaller than this, the test is not built to see.
How rarely you are willing to call a win that is not there.
How often you want to catch a real win when there is one. The half most calculators leave out.
Each extra challenger is another chance at a false positive, so each comparison has to clear a higher bar.
Total traffic entering the test each day, split across every variant.
Visitors needed per variant
31,234
62,468 in total, to detect 5% rising to 5.50%
[z(α) × √(2p̄q̄) + z(β) × √(p₁q₁ + p₂q₂)]² ÷ (p₂ − p₁)²
- Days to run
- 32
- longer than most tests survive
- total visitors ÷ daily visitors
- Effect targeted
- 10.0%
- 0.50 percentage points
- (target − baseline) ÷ baseline
- z critical
- 1.960
- two-sided
- Φ⁻¹(1 − α÷2)
- z power
- 0.842
- the power you asked for
- Φ⁻¹(power)
32 days is a long test. Past about a month, seasonality and site changes start contaminating the result, and the test is likely to be abandoned or peeked at. Look for a larger effect, accept lower power, or test somewhere with more traffic.
What a different target would cost
| Lift to detect | Per variant | Days |
|---|---|---|
| 2% | 7,52,703 | 753 |
| 5% | 1,22,124 | 123 |
| 10% ← yours | 31,234 | 32 |
| 20% | 8,158 | 9 |
| 50% | 1,471 | 2 |
Sample size scales with the inverse square of the effect. Halving the lift you are hunting for does not double the traffic you need — it quadruples it.
Fix this number before you start and do not look at the result until you reach it. Checking a test repeatedly and stopping the moment it crosses significance is the most common way false positives get shipped: a test peeked at daily will cross 95% by chance far more than 5% of the time.
How this is calculated
Visitors needed in EACH variant n = [ z(α) × √(2p̄q̄) + z(β) × √(p₁q₁ + p₂q₂) ]² ÷ (p₂ − p₁)² p₁ current conversion rate p̄ = (p₁ + p₂) ÷ 2 p₂ rate you want to detect q = 1 − p z(α) two-sided: Φ⁻¹(1 − α÷2) one-sided: Φ⁻¹(1 − α) z(β) Φ⁻¹(power) The first term is the noise the null hypothesis produces, the second the noise the alternative produces. A test is run under one and has to have power under the other, so both belong. Sample size scales with the INVERSE SQUARE of the effect: halve the lift you are hunting for → four times the traffic With k challengers against one control, α is divided by k (Bonferroni), so every extra variant costs traffic in every arm.
How it works
Every A/B test is a bet that a difference you can see in the data is a difference that exists in the world. Sample size is the price of that bet. Too little traffic and a real improvement looks like noise; too little traffic and noise looks like a real improvement. This works out how much you need before you start, which is the only time the number is any use.
Two thresholds go into it and most calculators only ask for one. Confidence is how rarely you are willing to call a win that is not there — 95% means a one-in-twenty false-positive rate. Power is how often you want to catch a real win when there is one, and 80% is the usual choice, meaning you will miss one real improvement in five. Leaving power out quietly assumes 50%: a coin flip.
The relationship that matters is that sample size scales with the inverse square of the effect. Detecting a 10% lift needs four times the traffic of a 20% lift, and a 5% lift needs sixteen times. This is why so many tests are abandoned halfway: they were never going to finish, and nobody worked that out first. The table on the result shows the curve for your own numbers.
Testing several variants at once costs more than people expect. Each challenger compared against the control is another chance to find a difference that is not there, so three challengers at 95% confidence gives roughly a one-in-seven chance of at least one false positive rather than one in twenty. The correction here divides the significance threshold between the comparisons, which is why every arm needs more traffic than a straight A/B test would.
Worked example
A checkout page at 5%, hunting a 10% relative lift, 95% confidence and 80% power
- p₁ = 5% p₂ = 5% × 1.10 = 5.5%
- Absolute difference 0.5 percentage points
- z(α) two-sided at 95% = 1.960
- z(β) at 80% power = 0.842
- n = [1.960 × √(2 × 0.0525 × 0.9475)
- + 0.842 × √(0.05×0.95 + 0.055×0.945)]² ÷ 0.005²
- ---------
- n = 31,234 per variant, 62,468 in total
- At 2,000 visitors a day that is 32 days.
- Hunting a 20% lift instead needs roughly a quarter of it.
31,234 visitors per variant — and a month of traffic to get them
How to use it
- Enter the conversion rate the control gets today. Use a real figure from the last few weeks, not a target.
- Choose the smallest improvement that would actually change what you do. If a 3% lift would not be worth shipping, do not size the test to find one.
- Leave confidence at 95% and power at 80% unless you have a reason. They are the conventional defaults and moving them has a large effect on the traffic needed.
- Say how many variants you are running. Every extra challenger raises the bar for all of them.
- Enter your daily traffic to see how long the test has to run. If that number is longer than a month, go back and target a larger effect — that is the honest response, not running it anyway.
What this tool does not do
- This sizes a test on a conversion RATE — a yes-or-no outcome per visitor. Revenue per visitor, time on page and average order value are continuous measures and need a different calculation, usually based on their standard deviation.
- It assumes visitors are split evenly between variants and that each visitor is counted once. Uneven splits and returning visitors seen in both arms both change the answer.
- It assumes you look at the result once, at the end. Checking daily and stopping when it crosses significance invalidates the whole calculation — that is what sequential testing methods exist for, and this is not one.
- The normal approximation behind it becomes unreliable when the expected number of conversions per arm is very small. Below about 30 conversions in an arm, treat the answer as indicative.
- Business cycles are not modelled. A test that runs for ten days covers different behaviour on weekdays and weekends, so running for whole weeks is usually wiser than stopping the moment the count is reached.
- The Bonferroni correction applied to multiple variants is deliberately conservative. More sophisticated corrections exist and need less traffic; none of them make testing five variants cheap.
Frequently asked questions
Why does it want so much more traffic than I expected?
Because conversion rates are noisy and the effects people hunt for are small. Detecting a 10% relative lift on a 5% baseline means separating 5% from 5.5% — half a percentage point — reliably enough to bet on. At 95% confidence and 80% power that takes 31,234 visitors per variant. The number is not pessimistic; the usual expectations are optimistic.
What is statistical power, and why do you ask for it?
Confidence protects you from calling a win that is not there. Power protects you from missing a win that is. At 80% power you will detect four out of five real improvements of the size you specified and miss the fifth. A calculator that does not ask for power is implicitly assuming 50% — a coin flip on whether you notice a real result — which is why its numbers look so much friendlier.
What effect size should I pick?
The smallest one that would change your decision. If you would not ship a change for a 3% lift, do not pay for the traffic to detect one. Working backwards from what is worth acting on is the useful discipline here — picking a small number because it feels ambitious just produces a test you cannot afford to run.
Can I stop the test early if it looks significant?
No, and this is the single most common way false positives get shipped. A test peeked at repeatedly will cross 95% confidence by chance far more often than 5% of the time, because you are giving it many opportunities to wander across the line. Fix the sample size before you start, and look when you reach it.
Should I use a one-sided or two-sided test?
Two-sided, almost always. It asks whether the variant is different rather than whether it is better, which matters because you want to know if a change made things worse. One-sided needs less traffic, and choosing it after seeing which way the result is going is a well-documented way of manufacturing significance out of nothing.
Why does testing more variants need so much more traffic?
Two reasons stacking. Your traffic is split more ways, so each arm fills more slowly; and each extra comparison is another chance at a false positive, so the threshold each one must clear gets stricter. Three challengers at 95% confidence would otherwise give roughly a one-in-seven chance of at least one spurious winner.
What about revenue per visitor rather than conversion rate?
Different calculation. This one is for binary outcomes — a visitor converts or does not. Continuous measures like revenue depend on their spread as well as their average, and a handful of large orders can dominate the result, so they generally need more data than a conversion rate does, not less.
My test finished. How do I check the result?
Use the conversion rate calculator on this site, which runs a two-proportion z-test on the numbers you ended up with and tells you the confidence level. This page is the half that runs before a test; that one is the half that runs after.
Related tools
tell us
Last updated 7 September 2026.