Minimum sample for group comparison

Sample size per group to compare two proportions or means (power, MDE)

Minimum sample for comparison — sample size per group needed to detect a given difference (MDE) with specified power (e.g. 80%).

FAQ about minimum sample for comparison

Parameters

Proportion in group 1 (baseline), %

E.g. current conversion 20% → enter 20

Proportion in group 2 (expected), %

Expected conversion after changes. Difference with group 1 = MDE

Mean in group 1

Mean in group 2

Standard deviation (common or average)

Equal variances in groups assumed

Significance level (α)

0.05 (95%) 0.01 (99%)

Power (1−β)

80% 90%

Assumptions and limitations

Two-tailed test
Equal group sizes
For means — equal variances

Sample size per group

—

respondents

In SurveyNinja, NPS, CSAT and more are calculated automatically. Create a survey in 5 minutes and get real-time analytics.

Start free

Data is not stored — calculation runs in your browser.

Formulas

Two proportions

n per group ≈ (z_α/2 + z_β)² × (p₁(1−p₁) + p₂(1−p₂)) / (p₁−p₂)². z_α/2 for α=0.05 is 1.96, z_β for 80% power is 0.84.

Two means

n per group ≈ 2 × (z_α/2 + z_β)² × σ² / (μ₁−μ₂)². σ = common SD, (μ₁−μ₂) = difference in means (MDE).

Power and MDE

80% power — in 80% of cases the test will detect the given difference. MDE — minimum difference you want to detect. Smaller MDE requires larger sample.

When to use which mode

Proportions — for conversion, shares, % satisfied (A/B on conversion). Means — for continuous metrics: average order, time, scale score. After calculation check significance with a separate calculator.

Sample size planning examples

1 Proportions: A/B on conversion

Variant A: conversion 20%, variant B: 28% (8 pp difference). α = 0.05, 80% power.

Calculator gives n per group (on the order of hundreds). You recruit this sample in each group; then check significance in the A/B test calculator.

2 Means: when to use "two means" mode

Not conversion but a continuous metric: average order, time on site, 1–10 scale score.

Enter expected means (μ₁, μ₂), common SD σ and MDE. Calculator returns n per group for comparing means (t-test). Estimate σ from past data or pilot.

3 Small difference — large sample

Conversion A: 10%, B: 12% (MDE 2 pp). Detecting a small effect requires large n.

Calculator will show several thousand per group. If that volume is unattainable, increase MDE or lower power.

4 CSAT of two segments

Comparing % satisfied (top-2) for men vs women. Expected proportions 70% vs 80%, α = 0.05, 80% power.

Enter two proportions — calculator returns minimum sample size in each group for comparing proportions.

5 Strict criteria (α = 0.01, 90% power)

Lower α — fewer false positives; higher power — better chance to detect the effect.

n per group will be larger than with α = 0.05 and 80% power. Use for critical decisions.

6 Average order: two channels

Channel 1: avg order 5000, channel 2: 5500. σ ≈ 2000 (from past data). MDE = 500.

"Two means" mode: enter μ₁, μ₂, σ and MDE. Calculator returns n per channel for t-test.

Frequently asked questions

When do you need per-group sample for comparison?

When planning an A/B test or comparing two segments (e.g. conversion after campaign A vs B, CSAT for men vs women). You need to know how many respondents to recruit in each group.

What is power?

Probability that the test will detect a real difference. 80% power — in 20% of cases we may fail to detect the effect (type II error). Target power is usually 80% or 90%.

What is MDE?

Minimum Detectable Effect — the smallest difference you want to detect. E.g. conversion 20% vs 25% — MDE 5 pp. Smaller MDE requires larger sample.

Why two modes — proportions and means?

Proportions — shares (conversion, % satisfied). Means — continuous metrics (average order, time, scale score). Formulas differ.

Are group sizes equal?

The calculator assumes equal groups (n in each). If you plan 2:1, take the calculated n for the smaller group; for the larger multiply by the ratio.

How to estimate standard deviation for means?

From past data or a pilot survey. If unknown — rule of thumb: for a 1–10 scale σ ≈ 2–3; for response time — from logs.

How does this differ from the single-proportion sample size calculator?

That one is for precision of a single proportion (margin of error, confidence interval). This one is for power to detect a difference between two groups. Different tasks.

After calculation how to check significance?

Once you have data, use the A/B significance calculator or z-test for two proportions/means. This calculator only plans sample size before data collection.

All calculators

Automate your metrics