Screenout

You launched a survey for dog owners. Of the 500 people who opened the questionnaire, 200 don't own a dog. If you let them all through, the data gets diluted and the conclusions become irrelevant.

Screenout is the mechanism that ends the survey for people who don't match the sample criteria, and it does so on the early questions - without wasting their time or yours.

Definition

Screenout is the forced termination of a survey for a respondent who fails the qualification criteria at the screening stage. It happens after they answer one or more screening questions at the start of the questionnaire. The respondent gets a closing message (usually neutral - "thank you for your interest") and is not admitted to the main part of the survey.

Why you need screenout

Every study targets a specific target audience. Including people who don't fit means getting mixed data, where the target group is "diluted" with irrelevant answers. This reduces the accuracy of estimates, distorts averages and lowers statistical power.

Beyond data quality, screenout saves resources. If you pay a reward for every completed survey (for example, through a respondent panel), letting everyone go all the way through is expensive. A screenout on question 2-3 costs less than a full 20-question survey for an irrelevant person.

Finally, the quality of the answers themselves suffers: someone who isn't a target user answers questions about an experience they don't have - out of politeness, at random, or with minimal attention. Such answers are worse than garbage: they look real but are substantively empty.

How screenout works technically

Screenout is built on logic jumps: if the answer to a screening question doesn't match the set criterion, the respondent is sent to a special closing screen rather than to the next question of the main part.

The structure of a typical screening:

1. Question: "Do you have a dog at home?"
2. If "No" → screenout screen ("Thanks for your interest in the survey; it's meant for a different audience")
3. If "Yes" → next screening question or the start of the main part

The screenout screen matters from a UX standpoint: the person should understand that everything is fine, they just aren't a fit for this particular survey. Aggressive or hurtful wording ("You don't qualify") undermines trust in the brand. A neutral "This survey is intended for a different group of participants - thank you for taking the time" works better.

Screenout rate: what it is and how to read it

Screenout rate is the share of respondents filtered out at the screening stage, out of the total number who started the survey. If 1,000 people started the survey and 350 got a screenout, the screenout rate is 35%.

The normal range depends on the specifics of the study. For a broad audience without strict criteria it's 10-20%. For narrow segments (for example, "B2B directors at companies with turnover above $500M"), a 60-80% screenout is considered normal. If the screenout rate is unexpectedly low, the criteria may be too loose, or the questions are transparent and people are guessing the "right" answer.

A high screenout rate affects the completion rate and the cost of data collection. When planning a survey, it's important to estimate in advance what percentage of your audience will pass the screening - so you can correctly calculate the reach you need. If target respondents are 30% of the base and you need 300 completes, you'll have to invite about 1,000 people.

Screener vs screenout: what's the difference

Screener is the set of questions at the start of the questionnaire that determine whether a person meets the study's criteria. It's the tool.

Screenout is the event itself: the moment when a specific respondent fails the screener and drops out. It's the result.

A screener may contain 1-2 questions for simple criteria or 5-7 for complex qualification (age + region + occupation + purchase frequency + ownership of a specific product). The more criteria there are, the longer the screener and the higher the screenout rate. The balance between sample precision and collection cost is one of the key decisions when designing a research design.

Common mistakes when setting up screenout

Transparent questions. "Have you used our product in the past 3 months?" at the start of a survey obviously hints that you should answer "Yes." Some non-target respondents will give the desired answer for the reward or out of curiosity. The fix: word screening questions neutrally, giving no signals about the "right" answer, and sometimes add decoy options to the answer lists (non-existent brands, products) - for quality control.

Screenout too late. The screening question is in 7th place; the person has already spent 5 minutes. This is annoying and lowers trust. Screening should be the first - or at most the second - block.

Strict criteria without estimating real reach. A combination of 6 criteria can filter out 95% of the potential audience. Before finalizing the screener, estimate what percentage of the base will actually pass - especially if you're working with a limited panel.

Ignoring selection bias. Even with a correct screenout, those who agreed to participate may differ from those who declined or never opened the survey. This is a separate type of bias that screenout does not solve - it only filters by the set criteria.

Example: screenout in a B2B study

A company is studying the experience of making software purchase decisions. Target audience: IT directors or heads of development departments at companies with 50+ employees who took part in selecting software in the past 12 months.

A three-question screener:

• "How many employees are there at your company?" → fewer than 50 → screenout
• "What position do you hold?" → not an IT director / not a head of development → screenout
• "Have you taken part in selecting or purchasing software in the past 12 months?" → no → screenout

Of 2,000 invited through a panel: 800 failed on position, 400 on company size, 200 on involvement in purchasing. 600 people passed the screening. Screenout rate - 70%. For such a narrow segment this is expected, and it was accounted for when planning the volume of invitations.

Screenout and sample representativeness

Screenout is directly tied to representativeness: the job of screening is to obtain a sample that accurately matches the target population. But there's a subtlety here: people who agree to take surveys already differ from those who refuse. This is non-response bias - screenout does not eliminate it; it merely filters by the set criteria.

If the screening works correctly but the survey is distributed through a single channel only - for example, only via an email base of loyal customers - the sample will still be biased. Screenout filters out irrelevant people by formal attributes but does not compensate for systematic bias in the distribution channel. You need to keep this in mind when interpreting the data.

With a quota sample, screenout is used together with quotas: screening lets through only those who meet the criteria, while quotas cap recruitment for each subgroup. For example, you need 100 people from each of three age groups - once a group's quota is filled, new participants from it automatically get a screenout, even if they formally qualify.

Screenout in SurveyNinja

In SurveyNinja, screenout is configured through logic jumps: a screening question at the start of the questionnaire, with a jump on a specific answer leading to a separate closing screen. The text of this screen is configured separately from the main "thank you" screen - you can write a neutral message for those filtered out without changing the final screen for those who pass the survey.

If you need to collect exactly N target responses and then stop accepting them, configuring limits will help - the survey will close automatically once the required number of completed questionnaires is reached.

Screenout is the forced termination of a survey for respondents who fail screening. It ensures a clean sample and saves budget and time. Key metrics: the screenout rate and its impact on collection cost. It's configured through logic jumps with a separate closing screen.

FAQ

How is screenout different from dropout?

Dropout is when a person started going through the main part of the survey and quit halfway. Screenout is an interruption before the main part, at the qualification-questions stage. Dropout points to problems with the length or quality of the questionnaire. Screenout means the person didn't fit the study's criteria from the start.

What should you put on the screenout screen so as not to offend the respondent?

Neutral wording works best. Good: "Unfortunately, this survey is intended for a different audience - thank you for taking the time." Bad: "You don't qualify for this study." Avoid the words "you don't qualify," "you don't meet," "excluded." If the survey runs through a panel with a reward, clarify whether the person will get partial compensation for the screenout: this reduces disappointment.

Can you hide the fact of screening from respondents?

Partly - yes. Screening questions can be disguised as ordinary demographic questions ("What is your position?") or product-usage questions, without revealing what exactly is the selection criterion. You can't hide screening entirely - at some point some people will get a completion screen and will realize they were filtered out.

How do you calculate how many invitations to send out given screenout?

The formula is simple: the required number of completes divided by the share that passes screening. You need 200 completed questionnaires, the expected screenout rate is 60% (so 40% will pass screening) - which means you need to send 200 / 0.4 = 500 invitations. Plus, factor in refusals to participate and incomplete questionnaires - the real number of invitations is usually 2-3 times the target number of responses.

Does the screenout rate affect data quality?

A high screenout rate on its own doesn't worsen the data - it only means that a large part of the audience isn't the target. Data deteriorates if screenout is set up incorrectly: questions that are too transparent let irrelevant people through, screening that's too late wastes time and budget, and no screening at all mixes the target and non-target audiences into a single sample.