Normal distribution

A histogram of scale responses sometimes looks like a bell: most values cluster in the middle, with fewer toward the edges. Such a shape is called a normal distribution. It isn't merely "nice to look at": how closely your data resembles it often determines which analysis methods you can use.

Many formulas for confidence intervals, t-tests, and regression rely on an assumption of normality; when data depart strongly from a normal distribution, some conclusions may lose their validity or call for different methods.

At the same time, survey responses are far from always normally distributed: bounded scales (for example, 1-5), shares of "agree / disagree", ratings skewed toward an edge - all of these can produce asymmetry or a "clipped" look. That's why it's important to understand what a normal distribution is, when people invoke it, and how to check your data before applying methods that depend on it.

What a normal distribution is, in plain terms

Normal distribution - a symmetric, bell-shaped curve: there are most observations in the center (around the mean), and their share falls off smoothly to the left and right. It is defined by two numbers - the mean and the standard deviation (which sets how "spread out" the bell is). Such a curve is typically characterized by this: roughly two-thirds of values fall within the band "mean plus or minus one standard deviation", and the band "plus or minus two deviations" captures the overwhelming majority. A number of statistical procedures are built on the assumption that we are dealing with data like this, or close to it.

Put more simply: if you plot a histogram of such data, you get a symmetric "hill" with its peak in the center. The more your data differs from this (sharp asymmetry, two peaks, "clipped" edges), the more cautious you should be with methods designed for normality.

A quick example. Consider the question "Rate from 1 to 5": with a normal distribution you would see a peak in the middle (for instance, the most "3" responses, fewer "2" and "4", fewer still "1" and "5") and symmetric "tails". In real surveys it's often different: 5% "1", 10% "2", 15% "3", 35% "4", 35% "5". That's a right skew, with no bell - and formulas designed for normality are applied to such data with caveats or replaced with non-parametric methods.

Why it matters in surveys

• Confidence intervals for the mean. A narrow interval around the sample mean (accounting for the standard error and a tabulated multiplier) gives a correct estimate of uncertainty only if the distribution of the sample mean is close to normal - which, as the sample size grows, is ensured by the central limit theorem. On small samples and with a highly "ragged" distribution of the underlying data, such an interval can be markedly off.
• Comparing groups (t-tests). A two-sample t-test and similar tests assume normality of the distribution within groups (or a sufficiently large size, in which case the distribution of means is close to normal anyway). When non-normality is pronounced, non-parametric analogues are sometimes used (for example, Mann-Whitney).
• Regression. In classical linear regression it is assumed that the residuals (model errors) are normally distributed. When this is badly violated, conclusions about coefficient significance and confidence intervals may be incorrect; switching to generalized models or robust standard errors is possible.

Bottom line: a normal distribution is not a goal of "making data look pretty", but an assumption of certain methods. If data don't fit it, you either choose different methods, or lean on asymptotics (large samples), or explicitly note the limitations.

When checking is especially appropriate. It makes sense to look explicitly at the distribution with a small sample (loosely, fewer than 30-50 per group), when comparing two groups on a quantitative variable (a t-test), and before building a linear regression. With large samples and simple description (means, shares), the central limit theorem often "saves" you - sample means behave normally even when the underlying data are non-normal.

When survey data are usually not normal

Bounded scales. Responses on a 1-5 or 1-10 scale are bounded above and below. When the mean is close to 4 or 5, the distribution often "runs up" against the edge - there's no symmetric bell. The same goes for shares of "yes/no" or "agree/disagree". For such variables, normality is the exception rather than the rule.

Skew in one direction. Satisfaction often produces a skew toward high ratings (mostly "4" and "5", few "1" and "2"). The histogram is asymmetric - that's not a normal distribution. Likert scales and other ordinal scales often behave exactly this way.

Few observations. With a small sample even from a normal population, the sample distribution may look "ragged"; meanwhile, normality tests are underpowered. You shouldn't rely on the test alone - look at the histogram and the meaning of the variable.

That's why survey reports often state "methods robust to departures from normality were used" or "a non-parametric test was applied" - this is precisely an acknowledgment that data are rarely perfectly normal.

How normality is checked

Plots. A histogram shows whether there is a single peak in the center and whether the "tails" are symmetric. A quantile-quantile (Q-Q) plot compares your data with a theoretical normal distribution: points along a straight line indicate closeness to normality, while a noticeable bend or "tails" to one side indicate a departure.

Statistical tests. Shapiro-Wilk, Kolmogorov-Smirnov, and similar tests answer the question "can the sample be considered drawn from a normal population". The limitation: on large N the slightest discrepancy leads to rejection of normality, while on small N the tests are barely sensitive. It makes sense to rely on plots and the substance of the variable. Even with a formal "rejection" of normality by a test (for example, with 500 responses), the shape of the distribution may remain acceptable for a t-test - judge by the situation.

The role of sample size. The central limit theorem states: as the sample size grows, the sample mean behaves ever closer to the normal law, even if the underlying quantity (for example, a rating on a 1-5 scale) is not distributed that way. That's why, when computing intervals and tests for the mean with a solid N, the normality assumption is often considered satisfied "in the limit", without a rigorous check of every variable.

Why "normal"

The name is historical: it was thought that many natural and measurement quantities (height, measurement errors) group together exactly like this. In surveys and survey scales this is not guaranteed - but formulas for statistical significance, margin of error, and confidence intervals still often use the properties of the normal distribution. Knowing when data are close to it and when they are not helps you choose methods correctly and frame caveats in the report.

Common mistakes

Demanding normality "at all costs". Survey data are often non-normal by nature. There's no need to discard variables or fit transformations just for the sake of a pretty chart - you need to choose suitable methods (non-parametric, robust) or explicitly rely on large samples and the central limit theorem.

Relying on the test alone. A single normality test on large N will almost always yield a "rejection", while on small N it may "miss" strong non-normality. Always look at the histogram and the Q-Q plot.

Confusing normality of a variable with normality of residuals. In regression you check the normality of the model's residuals, not of the original variables. The original predictors may be distributed any which way.

Ignoring normality where it matters. If you build a confidence interval for the mean on a small sample (for example, N=25) and the data are clearly skewed or have outliers, the classical formula may give an inaccurate interval. In such cases the bootstrap, non-parametric intervals, or an explicit note about limitations are appropriate.

How this looks in SurveyNinja

SurveyNinja has no built-in normality check. The reports display means and shares by question - from these you can judge the shape of the distribution only approximately. For histograms, Q-Q plots, and tests, you export the data to CSV/XLSX and analyze it in Excel, R, Python, or another package. If you then build confidence intervals or a regression in an external tool, that's usually where the assumptions are checked too.

Practical recommendations

For describing the sample normality is not required: the mean, median, shares, and spread are computed for any data. Normality matters when you move on to inference: tests, confidence intervals, regression.

With a small sample and doubts about normality, non-parametric methods are preferable, or an explicit statement in the report that methods tolerant of departures from normality were used.

What to write in the report. In the methodology section one sentence is enough: how you accounted for the shape of the distribution - for example, "checked via histogram" or "methods not requiring strict normality were used". This way the client sees that the assumptions were not ignored.

Mean, median, and spread. For an "ideal" bell the mean coincides with the median and mode, and the standard deviation describes the spread. If, in your data, the mean and median diverge noticeably - that's a signal of asymmetry and a possible departure from the normal law. It always makes sense to look at the spread too: the same average rating may correspond to different distribution shapes. For details, see the articles on descriptive statistics and the standard deviation.

A normal distribution sets the assumptions for some statistical methods; in surveys, because of short scales and skewed responses, data often don't fit it. It's worth checking the shape of the distribution wherever your conclusions depend on it, and switching to robust or non-parametric procedures when necessary.