Product update
Improved Flow Layout
Reorder Pages, annotate the canvas, and navigate complex Flows at a glance.
Calculate statistical significance of your A/B test
Compare two variants from any A/B test and see whether the lift is real or noise.
Variant
Visitors
Conversions
Rate
A
1.00%
B
1.11%
Hypothesis
Confidence
Variant B is treated as the challenger. Lift is calculated relative to A. Conversions must be ≤ visitors in each row.
Not yet significant
The difference between A and B could be random chance.
At 95% confidence there isn't enough evidence to call a winner. Keep the test running to collect more data.
A — Conversion rate
1.00%
B — Conversion rate
1.11%
Relative lift
+10.55%
p-value
0.0481
Sampling distribution Chance only Critical region Observed
How it's calculated
01 · Test statistic
Two-proportion z-test
The conversion rates of A and B are pooled to estimate a standard error, then the distance between B and A is measured in standard errors. That distance is the z-score.
02 · p-value
Probability of noise
The p-value is the chance of seeing a difference this large if the variants were actually identical. The smaller it is, the harder it is to explain away as random.
03 · Threshold
Significance threshold
When the p-value falls below 1 − confidence (e.g. 0.05 at 95%), the result is significant. Two-sided tests are stricter and the right default for most A/B tests.
Stop second-guessing your tests.
Nami Experiments runs A/B and multivariate tests with adaptive traffic allocation, automatic significance monitoring, and connected analytics — across mobile, web, and CTV.