A/B Test Calculator
Compare two variants' conversion rates to see the lift, relative improvement, and likely winner.
Published by Blueprint Dynamics. Planning tools, not professional advice. Read the formula, assumptions, and examples below before relying on a result. Independent professional review is not claimed. Learn more about our methodology →
AllTemplates • Blueprint Dynamics • hello@bpdynamics.io. Educational worksheet; entries and assumptions are not independently verified. This document is not tax, legal, medical, investment or engineering approval. Review current official rules and seek a qualified specialist where appropriate.
Scope and reference context
Use consistent cohorts and attribution windows. LTV and significance depend on assumptions; a normal-approximation A/B screen is not a sequential test or a guarantee of future performance.
Primary reading context, not an endorsement or evidence that the linked organization reviewed this tool. No independent professional sign-off is claimed.
About the A/B Test Calculator
A/B testing replaces opinion with evidence: show variant A to half your audience and variant B to the other half, then let their behavior decide which wins. The hard part is not running the test — it is knowing whether the difference you see is real or just random noise. A page that converts 6% versus 5% looks like a win, but with small samples that gap can vanish on a rerun.
This calculator compares two variants' conversion rates and quantifies the difference three ways: absolute lift in percentage points, relative lift as a percentage, and an approximate z-score that flags whether the result is statistically significant at the common 95% confidence level. It is built for marketers and product teams who want a quick, honest read on an experiment before rolling out a change to everyone.
How the A/B Test Calculator works
For each variant you enter visitors and conversions; the rate is conversions divided by visitors. Absolute lift is rate B minus rate A, expressed in percentage points (pp). Relative lift is that difference divided by rate A, which answers 'how much better, proportionally, is B?' — a far more intuitive way to describe improvement than raw points.
Significance uses a two-proportion z-test. The tool pools both groups' conversion rate, computes the standard error from that pooled rate and the two sample sizes, and divides the observed difference by that error to get a z-score. A magnitude of 1.96 or more corresponds to roughly 95% confidence that the difference is not due to chance. Below that threshold, the honest answer is to keep collecting data.
Worked example
Suppose Variant A drew 5,000 visitors and 250 conversions (a 5.0% rate) while Variant B drew 5,000 visitors and 300 conversions (a 6.0% rate). Absolute lift is 6.0% − 5.0% = 1 percentage point. Relative lift is (0.06 − 0.05) / 0.05 = 20%, so B converts a fifth better than A.
The pooled rate is 550 / 10,000 = 5.5%. The standard error is the square root of 0.055 × 0.945 × (1/5,000 + 1/5,000) ≈ 0.00456, giving a z-score of 0.01 / 0.00456 ≈ 2.19. Because 2.19 exceeds 1.96, the difference clears the 95% confidence bar and B can be declared the winner with reasonable confidence.
Tips & common mistakes
- Decide your sample size before you start, and do not stop the moment a result crosses significance — 'peeking' inflates false positives dramatically.
- Report relative lift for stakeholders but track absolute lift for revenue math; a 20% relative gain on a 1% base is only 0.2 points of extra conversion.
- Run tests for full weeks to average out day-of-week effects; weekday and weekend visitors often behave very differently.
- A z-score just under 1.96 is not a loss — it usually means you need more traffic, not that B failed. Keep the test running.
- Split traffic randomly and simultaneously. Comparing this week's B against last week's A introduces seasonality that no significance test can correct.
A/B Test Calculator: frequently asked questions
What does the z-score of 1.96 threshold mean?
It corresponds to a 95% confidence level (a 5% chance of a false positive) for a two-sided test. If the absolute z-score is 1.96 or greater, the observed difference is unlikely to be random noise, so you can reasonably trust the winning variant.
What is the difference between absolute and relative lift?
Absolute lift is the raw gap in percentage points — 6% minus 5% is 1 pp. Relative lift expresses that as a proportion of the baseline — 1 pp on a 5% base is a 20% improvement. Both matter: relative is intuitive, absolute drives the revenue calculation.
Why is my result not significant even though B looks better?
Usually it means your sample is too small to rule out chance. With few conversions, even a sizeable-looking gap can be noise. The fix is more traffic and time; the z-score will climb as evidence accumulates if the effect is genuine.
How long should I run an A/B test?
Long enough to reach your planned sample size and to cover at least one or two full weeks so weekday and weekend patterns are represented. Ending early or on a single strong day is the most common way teams fool themselves.
Can I test more than two variants here?
This calculator handles a head-to-head A versus B comparison. For three or more variants you would compare pairs, but be aware that testing many variations at once raises the chance of a false positive and generally needs more traffic per arm.