Statistical Power Calculator
Calculate the probability of detecting a true effect in your A/B test
When to use this calculator
Use this tool when:
- •You have fixed sample size constraints
- •You need to know what effects you can detect
- •You're planning experiment feasibility
Common scenarios:
- •Limited monthly traffic
- •Fixed experiment duration
- •Resource allocation decisions
Configure Your Test
Quick Presets
Target: 10.0% → 11.0%
Statistical Power
0%
Very low power
High risk of missing true effects
Detection probability
0.0% chance to detect a 10% lift
Type II error risk
100.0% chance of false negative
What Changes Can You Reliably Detect?
Power Analysis by Effect Size
Based on your sample size, here's your chance of detecting different improvement levels (effect sizes):
Your test: Looking for a 10% improvement → 0% chance of finding it
80% Power
Need ≥50% lift
Current Setup
10% lift = 0% power
Sample Size
n=5,000 per group
Recommendations
To achieve 80% power, you could:
- →Increase sample size: With a 10% effect, you'd need approximately ∞ users per group
- →Target larger effects: With 5,000 users, you can reliably detect effects ≥50%
- →Accept lower confidence: Though your current power is quite low for reliable decisions
Technical Details & Calculations
Current Parameters
Control (p₁)
10.00%
Treatment (p₂)
11.00%
Absolute Diff (Δ)
1.000%
Configuration
n=5000, α=0.05
Formula: Power = P(reject H₀ | H₁ is true)
Using normal approximation for proportions with pooled variance under H₀
When Power Matters Most
- •Product launches: Avoid missing performance regressions
- •Algorithm changes: Detect both improvements and degradations
- •Limited traffic: Know your detection limits upfront