Margin of error

Imagine this scenario: you run a customer survey and get the result "68% are willing to recommend the company to friends". A manager looks at the figure and asks, "If we ran this survey again, would it still be 68%?" The answer is almost certainly no. Even if everything is done perfectly, the result will "drift" a little around the true value.

This difference between the measured value and reality is not always a sign of poor research. Part of the spread is unavoidable simply because you are not working with the entire population, but only with a sample. This unavoidable inaccuracy is exactly what the margin of error describes.

Definition in plain terms

Margin of error (Sampling Error) is the statistical inaccuracy that arises because we survey not everyone, but only part of the audience. It is usually expressed as a range around the estimate: for example, 68% ± 3%, where ±3% is the margin of error.

Intuitively, the margin of error shows how much the result could change if you randomly selected a different group of respondents of the same size from the same population. The larger the sample and the more uniform the answers, the narrower this range.

The link with the confidence interval

In reports you will often see a note like "68% at a 95% confidence level and a ±3% margin of error". Behind it lies the concept of the confidence interval.

The interval "from 65% to 71%" is precisely the confidence interval, and "±3%" is its half, that is, the margin of error. The confidence level (95%) means: if you repeated the study many times on new random samples of the same size, then in roughly 95 cases out of 100 the true value in the population would fall within this interval.

An important takeaway: the margin of error does not say that "the result is wrong by 3%". It says that you honestly acknowledge a range of possible values and do not pretend that a single number reflects reality with absolute precision.

What the margin of error depends on

Sample size. The more valid answers you collect, the smaller the random spread and the narrower the confidence interval. That is exactly why mass public opinion polls use hundreds and thousands of respondents - to reduce the influence of chance.

Variability of answers. If the audience's opinion is sharply divided (say, 50% "for" and 50% "against"), the spread will be higher than in a situation where 90% agree with one option. Put simply, the more "noise" there is in the answers themselves, the larger the margin of error.

Confidence level. At a 99% confidence level the interval is wider than at 95%: you want to be more certain, so you allow a larger range of values. It is like choosing a "tight" or "loose" corridor for the true value.

In practice these parameters are taken into account in the calculations, and the researcher gets one clear figure - "the margin of error in our survey is about ±3.5%".

Intuitive examples

Suppose you run the same survey five times in a row on samples of the same size and composition. Each time you get slightly different percentages: 66%, 69%, 70%, 67%, 68%. None of them is "true" in the strict sense, but all of them lie within roughly the same range.

The margin of error describes exactly this range. If the calculations show that, given the chosen parameters, it equals ±3%, you can phrase the conclusion as: "The share of satisfied customers is, with high probability, somewhere between 65% and 71%." This is more honest and more useful than trying to prove that the "correct" figure is exactly 68%.

The same logic matters when comparing monitoring waves or segments within a single survey. A difference between 68% and 70% with a ±3% margin of error may simply be a consequence of random variation rather than a real change in the situation.

The margin of error and other types of error

It is important to distinguish the margin of error from other sources of inaccuracy, which are statistical deviations in surveys that go beyond random sampling variation.

Coverage error. Part of the audience is not included in the sample to begin with (for example, you have no contacts for customers above a certain age). This is a systematic bias that is not compensated for by increasing the sample.

Nonresponse error. Even among those invited, not everyone takes part. If the respondents are mostly the more loyal or the more dissatisfied customers, the overall result will be biased, regardless of how small your formal margin of error is.

Measurement errors. Unclear question wording, leading answer options, technical problems - all of this produces systematic distortions that are not reflected in the "±3%" figure at all.

Conclusion: a small margin of error does not yet guarantee fully accurate results. It only says that you control the random component reasonably well. The other sources of error must also be taken into account when planning and interpreting surveys.

How to interpret the margin of error in reports

Compare intervals, not just percentages. If one segment shows 60% ± 4% and another shows 65% ± 4%, their intervals (56-64% and 61-69%) overlap. This means you cannot confidently say that the second segment is "significantly better" than the first - the difference may be random.

Look at the sample size within segments. An overall sample of a thousand people will not save you from a high margin of error if an important subgroup contains only 40 respondents. For a correct comparison of segments, the amount of data for each one needs to be comparable.

Do not overrate small shifts. A change in a metric from 62% to 64% with a ±4% margin of error almost never indicates a real improvement. Much more important are sustained changes by an amount that substantially exceeds the margin of error itself.

Practical recommendations

Discuss the margin of error at the task-setting stage. Instead of an abstract "we need to run a survey", formulate the requirement: "We will be satisfied with a margin of error of about ±5% at a 95% confidence level for the key metric." This helps you understand right away what sample size and budget are needed.

State the margin of error and confidence level in reports. A single line in the methodology block is enough for colleagues to understand how reliable the figures are. This is good practice for any quantitative research and an important element of transparency.

Combine quantitative and qualitative methods. The margin of error describes only numerical estimates. To understand the reasons behind changes, it makes sense to complement quantitative surveys with qualitative methods - these are covered in materials on quantitative research and in the Qualitative Research term in the glossary.

Watch the quality of the sample, not just its size. Increasing the number of respondents reduces the formal margin of error, but does not fix biases caused by an incorrect audience selection. It is better to have fewer respondents that are more accurate and closer to the target structure than many with a systematic bias.

The margin of error is not an enemy but an honest acknowledgment of the limits of your data. Once you learn to calculate and interpret it correctly, discussing survey results stops being an argument about the "right number" and turns into a conversation about ranges, probabilities, and informed management decisions.