🧪

A/B Test Duration Calculator

Calculate how long to run experiments for statistical significance

Baseline Conversion Rate (%)

Minimum Detectable Effect (%)

e.g., 20% means detecting 5% → 6% change

Daily Visitors Per Variant

Statistical Significance

Statistical Power

Sample Size Per Variant

8,162

Total Sample Needed

16,324

Estimated Test Duration

9 days

About 1.3 weeks

What This Means

You need 8,162 visitors per variant to detect a 20% relative change (5% → 6.00%)

With 95% confidence, there's only a 5% chance of a false positive.

With 80% power, you have a 80% chance of detecting a real effect if it exists.

Quick Reference

5% MDE

123d

10% MDE

32d

20% MDE

A/B Testing Best Practices

• Don't peek at results early - it inflates false positive rates
• Run tests for full weeks to account for day-of-week effects
• Test one change at a time for clear attribution
• Larger effects are easier to detect (need less traffic)

What is A/B Test Duration Calculator?

An A/B test duration calculator determines the sample size and time required to achieve statistically significant results from a controlled experiment. It accounts for baseline conversion rate, minimum detectable effect (MDE), statistical power, and significance level to prevent premature or inconclusive test conclusions.

How to use this calculator

1Enter your baseline conversion rate — the current rate before any changes.
2Set your minimum detectable effect: the smallest relative improvement worth detecting (e.g., 10%).
3Adjust statistical power (80% is standard) and significance level (95% or 99%).
4Review the required sample size per variant and estimated test duration based on your daily traffic.
5Use the duration to decide if the test is worth running or if you need to test a bolder change.

Why this matters for founders

Ending A/B tests too early leads to false positives -- you ship changes that do not actually improve anything. Running them too long wastes traffic and delays decision-making. Proper test duration planning ensures you make real, data-driven product decisions.

How to read the results

•A standard test uses 95% confidence (alpha = 0.05) and 80% statistical power (beta = 0.20) -- these are the defaults for good reason.
•Smaller minimum detectable effects require larger sample sizes and longer test durations -- detecting a 1% lift takes far more data than detecting a 10% lift.
•Never peek at results and stop early when the variant 'looks good' -- this dramatically inflates false positive rates.
•If the required duration exceeds 4-6 weeks, consider whether the test is worth running or if you should test a bolder change.

Industry benchmarks

Most landing page tests need 1,000-10,000 visitors per variant. Detecting a 5% relative improvement on a 3% baseline conversion rate typically requires ~25,000 visitors per variant. Industry standard is 95% confidence and 80% power. Average test duration for mid-traffic sites is 2-4 weeks.

Key terminology

🧪A/B Testing