Systematic sampling

Imagine the following situation: you have a database of 20,000 customers and you need to survey roughly 1,000 of them. Picking each respondent randomly, one by one, is slow and inconvenient. A researcher suggests a simple trick: sort the customers by contract number and take every twentieth one. The result is a neat, "even" sample without any complicated math. But can you trust it?

If there is some hidden periodicity in the customer list (for example, tied to promotion dates or acquisition channels), systematic selection may regularly "land" on the same subgroup and produce a distorted picture. That is why it is important to understand how systematic sampling works, when it is appropriate, and what pitfalls it has.

Definition of systematic sampling

Systematic sampling is a way of building a sample in which elements of an ordered population are selected at a fixed interval: for example, every tenth or every twentieth element of a list, starting from a randomly chosen starting point.

In essence, this is a "simplified" version of random sampling: you choose the start randomly just once, set a step k, and then automatically collect the required number of respondents. This approach saves time when running a separate random selection for each element is inconvenient.

How systematic sampling is built in practice

1. A list of the population is compiled. For example, all customers from the past year, all order numbers, all records in the CRM. It is important that the list be complete and not contain excluded groups.

2. The elements are ordered. Most often by time, contract number, or customer ID. At this stage it is easy to introduce periodicity without noticing: for example, if contract numbers depend on the branch office and you sort by number alone.

3. A step k is chosen. If you need to survey roughly N people from a list of length M, the step is calculated as M/N and rounded to the nearest whole number. With a list of 20,000 elements and a plan for 1,000 respondents, the step will be 20.

4. A random starting position is determined. For example, a random number from 1 to k. Selection begins from that element: the first respondent is position 7, the next is 27, then 47, and so on, until the required sample is collected.

Pros and cons of systematic sampling

Advantages. The main strength is the simplicity and speed of implementation. You only need to calculate the step and the start once, and after that the procedure is easy to automate. In even, "randomly" ordered lists, such a sample yields results very close to simple random sampling.

Limitations. The main risk is hidden periodicity in the list. If, say, orders from offline stores and the online channel alternate by number, and your step always lands on orders from a single channel, you will get a sample with a distorted structure. Another problem is the lack of control over important subgroups: systematic selection on its own does not guarantee that the sample will contain enough representatives of each segment.

Systematic sampling in time-based research

A special case often used by marketers and product teams is selecting "every n-th event" over time. For example, you decide to survey every hundredth customer who places an order in a store, or every fiftieth user who completes a flow in an app.

This approach is convenient for building time series and monitoring metrics: you can regularly measure satisfaction, NPS, or other metrics without having to survey every customer. Later, these data can be analyzed as a time series (the term Time Series Analysis explains more about working with such structures).

It is important that the choice of steps and time points does not coincide with periods when customer behavior differs sharply (for example, only weekdays, only the first half of the day, or only sale days), otherwise you risk introducing a persistent bias.

Example: a survey at offline points of sale

Suppose you want to collect feedback in a retail network or at an event. One working technique is to survey not every visitor, but, say, every tenth one. This reduces the load on staff and customers while still providing a regular stream of data. A poster with a QR code leading to the survey is placed at the counter, checkout, or booth, and employees invite people to fill out the form according to a predefined rule.

You can read more about how this format works in the article "QR code: an offline approach": it covers the specifics of placing codes and scenarios for working with printed materials. Combined with systematic selection (for example, inviting every n-th visitor), this helps maintain a steady stream of responses without overwhelming the audience.

For events and conferences, the ideas in the article "Surveys at events" are useful: it covers ways to embed a survey into the event flow without disrupting the participant's main experience. Systematic selection (every fifth session attendee, every tenth booth visitor) helps obtain a more even picture than random "spikes" of activity at the beginning or end of the day.

How to use systematic sampling in online surveys

Selecting contacts in a CRM. It is often easier to export one large list of customers, sort it by time or ID, and then select contacts by step k. These people are then sent an invitation to the survey. This approach is especially convenient in large projects estimating market size or analyzing the customer base, when you are dealing with tens of thousands of records (see the article "Estimating market size").

Pseudo-random selection of website visitors. Online, "every n-th" visitor can be implemented technically through counters, triggers, and segments. For example, you can show a survey invitation not to everyone, but only to a portion of the audience, so as not to overload users. Here it is important to ensure that the display mechanism does not depend on the traffic channel or the time of day, otherwise the sample will become biased.

Controlling the structure of responses. Even if you initially selected respondents systematically, it is useful in reports to check how the answers were distributed across key parameters. In SurveyNinja you can use filters and breakdowns in reports for this, as well as the settings described in the help section on filtering responses ("Viewing the survey report").

Common mistakes

Ignoring the structure of the list. If the list elements are ordered by a characteristic related to the metric under study, systematic selection can give a heavily distorted picture. For example, when customers in the CRM are ordered by manager, and each manager works with a different segment.

A step that matches a periodicity. A classic example: in a store every seventh purchase is part of a weekend promotion, and you choose a step of 7. As a result, the sample contains a disproportionate number of "promo" customers, and the survey results are distorted.

No random start. If you always begin selection from the first element rather than from a random point within the first interval, you make things harder for yourself: any peculiarities of the first records will systematically affect the sample.

Blind trust without verification. Even if the selection scheme looks neat, after collecting the data it is useful to compare the structure of the sample with the structure of the whole population across available characteristics: region, customer type, traffic source. The reports and segmentation tools described in the article "Segmenting and filtering customers through surveys" are handy for this.

Practical recommendations

Don't use systematic sampling "by default." It is a convenient tool, but it should be a conscious choice. First assess how your list is structured, whether it has pronounced groups or cycles, and only then choose the step and the start.

Combine it with other methods. It is often reasonable to use systematic selection within a larger design: for example, first split the audience into several segments, and then apply a systematic scheme within each segment. This helps you control the structure of the sample more effectively.

Describe the methodology honestly. In research reports you should explicitly state that the sample was systematic, exactly how the list was compiled, what step was used, and how the starting point was chosen. This will allow the results to be interpreted correctly and compared with data from other sources.

Use the survey builder's features to control the load. In SurveyNinja you can limit the total number of responses, set up different links for different channels, and apply response filtering in reports. All of this helps implement the ideas of systematic selection more carefully, without overwhelming the audience and while keeping the fieldwork stage manageable.

Systematic sampling is a tool for those who want to make life easier for field teams without mindlessly sacrificing data quality. When applied carefully, it produces results close to random sampling, but it requires a little more attention to how your database is structured and where cycles and patterns may be hiding within it.