Non-response bias

A satisfaction survey was sent to 1,000 customers; 200 replied. If the non-respondents include more dissatisfied people (they simply didn't feel like spending time on the survey), the average score among respondents will come out inflated. This is non-response bias: those who didn't respond differ systematically from those who did, and the results stop reflecting the population. Non-response bias is a subtype of bias and is closely related to selection bias.

Unlike response bias (distortion of answers that were actually given), non-response bias arises before the answer: some of those invited don't take part at all. The lower the response rate, the higher the risk that the "silent" group is special and that conclusions about the sample will be biased.

What non-response bias means in plain terms

Non-response bias is a systematic difference between those who did not respond to a survey and those who did. Results are built only from respondents; if non-respondents differ on important traits (opinion, behavior, demographics) from respondents, the sample estimates shift relative to the population. Non-response bias grows stronger with a low response rate and with systematic reasons for non-response (refusal, unavailability, lack of interest).

Put simply: non-response bias is when "those who stay silent aren't like those who answered." If the ones not responding are mostly the dissatisfied or, conversely, the busiest people, the averages and proportions among respondents stop reflecting everyone.

Why people don't respond

No time or interest. Respondents put off the survey or close it without starting. Often the busy or those who don't relate to the topic don't respond, and this group may differ from respondents.

Inconvenient channel or moment. The survey arrived by email but the person rarely checks their inbox; the survey appeared at an inconvenient moment (work, travel). Such people are systematically dropped from the sample.

Trust and anonymity. If a respondent doesn't trust the organizer or isn't sure about confidentiality, they may not respond. The role of anonymity in surveys is a factor here.

Survey length and complexity. Long or overloaded surveys are abandoned more often. Those who leave may differ from those who reach the end (for example, more patient or more interested).

Question content. Questions that are too personal, boring, or irrelevant raise refusal rates. Particular segments (for example, by age or experience) may drop out more often than others.

When non-response bias is stronger

Low response rate. At 10-20% response the risk of bias is high: most people "stay silent," and we don't know how they differ. At 60-70% and above, non-response is usually closer to random.

A single collection point. If the survey runs only by email or only through one channel, you lose those who don't use that channel. A variety of channels and an email distribution with reminders help raise the response.

A single appeal with no reminders. One email or one message yields a low response; some non-respondents simply "didn't see it" or "forgot." Reminders increase the share of respondents and can bring it closer to the population.

Voluntary participation without incentives. With purely voluntary participation, the most motivated or dissatisfied tend to respond more often, which amplifies non-response bias.

Examples of non-response bias

Post-purchase customer survey. Those who are disappointed or in a hurry respond less often, so satisfied customers are over-represented among respondents. The average NPS or CSAT is inflated.

Employee survey. The busy, the skeptics, or those who don't believe in anonymity don't respond. The sample may over-represent the "convenient" or more loyal employees, and the overall company picture is distorted.

Panel surveys. Over time, certain types of respondents drop out of the panel. If you ignore panel attrition and don't refresh the sample, non-response bias accumulates.

Online surveys of a target audience. If the target audience is heterogeneous and the survey is available only online and in one language, older people or groups that use the internet less are under-represented, which is both coverage bias and non-response bias.

How to minimize non-response bias

Raise the response rate. A short survey, a clear invitation, reminders, and a convenient channel and moment increase the response. The higher the share of respondents, the less weight the "silent" segment carries and the closer the sample is to the population.

Multiple channels and reminders. Email distribution plus reminders and, if needed, an alternative channel (a link in a messenger, on the site). This way you reach respondents with different behaviors.

A short survey and logic. Fewer questions and logic jumps reduce drop-off during the survey and make non-response less "selective."

Pilot and drop-off assessment. A pilot survey helps assess who drops out and at which question. Based on the pilot you can adjust the length and wording before the main collection.

Analysis of respondents and non-respondents. Where possible, compare respondents with known characteristics of the population (gender, age, region from the database). If respondents differ noticeably, the report should note the limitations and possible non-response bias.

Weighting. If the structure of the population and the structure of respondents are known, you can apply a weighted survey to bring the sample closer to the population. This does not remove non-response bias entirely, but it can correct part of the distortion.

Relationship to selection bias and representativeness

Non-response bias is a special case of sampling problems: we observe only respondents, not the entire sample or population. It directly affects representativeness: even with an initially random invitation, a low response and systematic non-response make the final sample unrepresentative.

In the report it helps to state the response rate, the distribution method, the number of reminders, and briefly who might not have responded and how that could have affected the conclusions. Understanding the respondent in a survey and their motivation to participate matters here.

Common mistakes

Ignoring a low response rate. Interpreting results at 15-20% response the same way as at 70%, without mentioning non-response bias. This creates a false impression of reliability.

Not sending reminders. A single appeal without reminders yields a low response and increases the risk that non-respondents are a special group.

A long survey without logic. Many required questions in a row increase drop-off; those who left may differ systematically from those who reached the end.

Not describing limitations. Failing to state the response rate in the report and not discussing who might not have responded and how that could have biased the results.

How this looks in SurveyNinja

In SurveyNinja you can set up invitation distributions and reminders, reducing the share of the "silent." Short surveys and logic jumps lower drop-off as people go through. Ready-made product evaluation templates and support surveys help you launch surveys with a well-considered length more quickly. In reports you can record the response rate and comment on the possible impact of non-response on the conclusions.

Practical recommendations

Track the response rate. Record the share of respondents out of those invited; with a low response, clearly note the risk of non-response bias in the report.

Use reminders. A single distribution rarely yields a representative response; 1-2 reminders noticeably increase the response rate.

Keep the survey short and relevant. Fewer questions and showing only relevant blocks via logic reduce selective drop-off.

Pilot before the main collection. Use the pilot to assess who drops out and where, and adjust the survey.

What to write in the report. State the response rate, the invitation and reminder method, and briefly who might not have responded and how that could have affected the interpretation.

Non-response bias is a systematic difference between non-respondents and respondents, because of which the sample results shift relative to the population. It is minimized by raising the response rate (a short survey, reminders, convenient channels), piloting, analyzing drop-off, and explicitly describing the limitations in the report.