Non-response bias

Non-response bias is the distortion that appears when people who do not answer a survey differ systematically from those who do, so the results stop reflecting the population. Researchers assess and reduce it to know how far survey estimates can be trusted. For example, a satisfaction survey goes to 1,000 customers and 200 reply; if more of the dissatisfied ones stayed silent, the average score among respondents comes out inflated. Non-response bias is a subtype of bias and is closely related to selection bias.

Unlike response bias (distortion of answers that were actually given), non-response bias arises before the answer: some of those invited don't take part at all. The lower the response rate, the higher the risk that the "silent" group is special and that conclusions about the sample will be biased.

What non-response bias means in plain terms

Non-response bias is a systematic difference between those who did not respond to a survey and those who did. Results are built only from respondents; if non-respondents differ on important traits (opinion, behavior, demographics) from respondents, the sample estimates shift relative to the population. Non-response bias grows stronger with a low response rate and with systematic reasons for non-response (refusal, unavailability, lack of interest).

Put simply: non-response bias is when "those who stay silent aren't like those who answered." If the ones not responding are mostly the dissatisfied or, conversely, the busiest people, the averages and proportions among respondents stop reflecting everyone.

Why people don't respond

No time or interest. Respondents put off the survey or close it without starting. Often the busy or those who don't relate to the topic don't respond, and this group may differ from respondents.

Inconvenient channel or moment. The survey arrived by email but the person rarely checks their inbox; the survey appeared at an inconvenient moment (work, travel). Such people are systematically dropped from the sample.

Trust and anonymity. If a respondent doesn't trust the organizer or isn't sure about confidentiality, they may not respond. The role of anonymity in surveys is a factor here.

Survey length and complexity. Long or overloaded surveys are abandoned more often. Those who leave may differ from those who reach the end (for example, more patient or more interested).

Question content. Questions that are too personal, boring, or irrelevant raise refusal rates. Particular segments (for example, by age or experience) may drop out more often than others.

When non-response bias is stronger

Low response rate. When most people stay silent, there is more room for them to differ from respondents. The response rate alone does not determine the bias, though: Groves and Peytcheva's 2008 meta-analysis in Public Opinion Quarterly found only a weak link between response rates and the size of non-response bias. What matters most is whether the reasons for not answering are related to what you measure.

A single collection point. If the survey runs only by email or only through one channel, you lose those who don't use that channel. A variety of channels and an email distribution with reminders help raise the response.

A single appeal with no reminders. One email or one message yields a low response; some non-respondents simply "didn't see it" or "forgot." Reminders increase the share of respondents and can bring it closer to the population.

Voluntary participation without incentives. With purely voluntary participation, the most motivated or dissatisfied tend to respond more often, which amplifies non-response bias.

Examples of non-response bias

Post-purchase customer survey. Those who are disappointed or in a hurry respond less often, so satisfied customers are over-represented among respondents. The average NPS or CSAT is inflated.

Employee survey. The busy, the skeptics, or those who don't believe in anonymity don't respond. The sample may over-represent the "convenient" or more loyal employees, and the overall company picture is distorted.

Panel surveys. Over time, certain types of respondents drop out of the panel. If you ignore panel attrition and don't refresh the sample, non-response bias accumulates.

Online surveys of a target audience. If the target audience is heterogeneous and the survey is available only online and in one language, older people or groups that use the internet less are under-represented, which is both coverage bias and non-response bias.

How to minimize non-response bias

Raise the response rate. A short survey, a clear invitation, reminders, and a convenient channel and moment increase the response. The higher the share of respondents, the less weight the "silent" segment carries and the closer the sample is to the population.

Multiple channels and reminders. Email distribution plus reminders and, if needed, an alternative channel (a link in a messenger, on the site). This way you reach respondents with different behaviors.

A short survey and logic. Fewer questions and logic jumps reduce drop-off during the survey and make non-response less "selective."

Pilot and drop-off assessment. A pilot survey helps assess who drops out and at which question. Based on the pilot you can adjust the length and wording before the main collection.

Analysis of respondents and non-respondents. Where possible, compare respondents with known characteristics of the population (gender, age, region from the database). If respondents differ noticeably, the report should note the limitations and possible non-response bias.

Weighting. If the structure of the population and the structure of respondents are known, you can apply a weighted survey to bring the sample closer to the population. This does not remove non-response bias entirely, but it can correct part of the distortion.

Relationship to selection bias and representativeness

Non-response bias is a special case of sampling problems: we observe only respondents, not the entire sample or population. It directly affects representativeness: even with an initially random invitation, a low response and systematic non-response make the final sample unrepresentative.

In the report it helps to state the response rate, the distribution method, the number of reminders, and briefly who might not have responded and how that could have affected the conclusions. Understanding the respondent in a survey and their motivation to participate matters here.

Common mistakes

Ignoring a low response rate. Interpreting results from a low response rate the same way as from a high one, without mentioning non-response bias. This creates a false impression of reliability.

Not sending reminders. A single appeal without reminders yields a low response and increases the risk that non-respondents are a special group.

A long survey without logic. Many required questions in a row increase drop-off; those who left may differ systematically from those who reached the end.

Not describing limitations. Failing to state the response rate in the report and not discussing who might not have responded and how that could have biased the results.

How this looks in SurveyNinja

In SurveyNinja you can share one survey through several channels at once (a link, a QR code, a website widget or pop-up), so people who rarely check email still get a chance to answer. Logic jumps show each respondent only the questions that apply to them, which keeps the survey short. Ready-made survey templates help you start with a survey of reasonable length.

Practical recommendations

Track the response rate. Record the share of respondents out of those invited; with a low response, clearly note the risk of non-response bias in the report.

Use reminders. A single distribution rarely yields a representative response. Don Dillman's Tailored Design Method builds on several contacts with each invitee, because reminders bring in people who missed or postponed the first message.

Keep the survey short and relevant. Fewer questions and showing only relevant blocks via logic reduce selective drop-off.

Pilot before the main collection. Use the pilot to assess who drops out and where, and adjust the survey.

What to write in the report. State the response rate, the invitation and reminder method, and briefly who might not have responded and how that could have affected the interpretation.

Non-response bias is a systematic difference between non-respondents and respondents, because of which the sample results shift relative to the population. It is minimized by raising the response rate (a short survey, reminders, convenient channels), piloting, analyzing drop-off, and explicitly describing the limitations in the report.