Probability sampling

Picture this situation: a city administration wants to learn how residents feel about a new transport reform. You could interview people near the nearest metro station and draw conclusions from those 300 interviews. You could post a questionnaire on the city hall website and wait to see who drops by and decides to speak up. Or you could plan in advance which districts, age groups and household types should be included in the study, and randomly recruit respondents from each segment.

In all three cases you end up with a table of percentages. But the level of trust in those numbers will differ. In the first and second options you are effectively surveying whoever happened to be "within reach" or chose to be active. In the third, you build the sample so that every resident has a non-zero and known probability of being included in the study. This is exactly the approach known as probability sampling.

Definition and the core idea

Probability sampling is a way of building a sample in which every element of the population has a known and non-zero probability of being selected. Thanks to this, you can formally estimate the margin of error, build confidence intervals and generalize the results to the entire population with a controlled degree of imprecision.

To put it more simply, in probability sampling randomness does not mean chaos. It means you have a list or clear selection rules under which any member of the group being studied can theoretically end up in the sample, and the probability of this can be described mathematically.

Why it is considered the "gold standard"

You can estimate the accuracy of your results. In a probabilistic design it is legitimate to talk about a formal margin of error and confidence intervals: this is exactly where the formulas described in the articles on the confidence interval and statistical deviations apply. In studies with a convenience sample or "whoever wants to, answers" surveys, such estimates will be highly approximate.

You control the representation of subgroups. With a well-designed sample, you make sure in advance that both large and small but important audience segments end up in the study. This is especially critical in sociological surveys and national research.

The results are easier to defend. When a client or an external audit asks "why do you think these 1,200 people represent all residents?", you have a formal answer: the described sample design, the calculation of selection probabilities and the estimate of the margin of error. This adds credibility to the study.

Main types of probability sampling

Simple random sampling. You have a complete list of the elements of the population — for example, a database of all customers. You select N records at random, so that every customer has an equal chance of ending up in the sample. Conceptually this is the simplest and most "honest" method, but in practice it is not always feasible: not everyone has complete and up-to-date lists.

Stratified sampling. The population is divided into homogeneous groups (strata): regions, age categories, customer types. From each stratum you randomly select respondents in proportion to their share in the population. This helps avoid a situation where, say, residents of large cities happen to be heavily overrepresented compared with smaller ones.

Cluster sampling. Instead of randomly selecting individual people, you randomly select clusters: schools, buildings, companies, stores. You then survey everyone within the selected clusters or a random portion of them. This approach saves resources when the population is heavily "spread out" across a territory.

Systematic sampling. The elements of the population are arranged in a list (for example, by purchase time or by contract number), after which you select every k‑th one: every tenth, twentieth, and so on. The method is simple to implement, but it requires caution: the list must not contain a hidden periodicity that coincides with your step.

Probability vs non-probability sampling: what the difference looks like in practice

In real surveys, researchers often use mixed schemes. For example, they start with a probabilistic design and then run into incomplete contacts, refusals to participate and other limitations. As a result, the "ideal" scheme partly turns into a convenience sample.

Non-probability methods — convenience, quota, the "snowball" sample (Snowball Sampling) — are often indispensable when you are working with hard-to-reach groups or a limited budget. But it is important to call them by their proper names honestly and not attribute to them the degrees of accuracy characteristic of strict probabilistic designs.

It is considered good practice to at least partly bring the "field" reality closer to a probabilistic scheme: to control the composition of respondents on key characteristics, to track which groups are underrepresented, and, if necessary, to recruit them deliberately.

How probability sampling works in online surveys

In the digital environment, the idea that "everyone has a known probability of ending up in the sample" sounds more complicated than in the classic textbook examples, but the principles remain the same.

Your own databases. If you have a complete customer database, you can theoretically select people from it at random and send them invitations. Then the response rate comes into play: the lower the response, the more the resulting design departs from the ideal probabilistic scheme.

Respondent panels. Specialized respondent panels let you set audience parameters and the sample size. Inside the panels, their own probabilistic and quasi-representative recruitment schemes are applied, which helps you get closer to the "gold standard" without having to build your own panel from scratch.

Data weighting. When perfectly probabilistic selection is unattainable, subsequent adjustment comes to the rescue. Weighting techniques (for more on them, see the term Weighted Survey) make it possible to adjust the contribution of answers from different groups so that the sample better matches the structure of the population.

When you cannot do without probability sampling

Not every survey needs a complex design. For quick marketing studies, concept testing or UX surveys, a carefully planned convenience sample is often enough. But there are situations where the risk of going without a controlled probabilistic scheme is too high.

National and city surveys. When the results reach the media, influence policy or important public decisions, the requirements for representativeness and transparency are extremely high. Here probability sampling is not so much "desirable" as "mandatory".

Long-term tracking studies. If you regularly measure the same indicators (brand awareness, trust, satisfaction), it is important that the difference between waves reflects real changes rather than the fact that you recruit respondents differently each time. A standardized probabilistic design helps reduce this risk.

Studies with a high cost of error. When survey results directly affect major investments, new product launches or changes in how an organization works, it makes sense to invest in a stricter sample design rather than cut corners at the stage that sets the quality of all subsequent conclusions.

Practical recommendations

Start by describing the population. Clearly state who you want to draw conclusions about: "all customers over the past year", "city residents over 18", "app users who have paid for a subscription". Without this, you cannot design a probabilistic — or any other meaningful — sample.

Choose the strictest design that fits your resources. If there is no complete list of the population, think about which approximations of it are available: registries, partner databases, panel services. The closer you are to a probabilistic scheme, the easier it will be to defend your results.

Document the respondent recruitment scheme in your methodological description. Even if the actual design is far from ideal, it is important to describe honestly and in detail exactly how you recruited people. This will help interpret the conclusions correctly and avoid attributing excess precision to the data.

Combine probabilistic approaches with qualitative methods. A strict sample answers the question "how often does something occur", but it does not always explain "why". The balance between quantitative and qualitative approaches is discussed in materials on quantitative research and in the term Quantitative Research.

Probability sampling is not an academic whim but a tool that lets you talk honestly about the accuracy of your numbers. The closer your real design is to this ideal, the fewer reasons there are to doubt the survey results and the more confident you feel when making decisions based on data.