Ranking (ranking question)

"Rank these from 1 to 5 by importance: price, quality, delivery, assortment, service" - the respondent not only marks what matters, but puts it in order. This is ranking: the order itself carries meaning.

Unlike multiple choice, where order doesn't matter, here every option gets a rank - 1st, 2nd, 3rd. In SurveyNinja it's implemented with the "Ranking" element - drag-and-drop or numbering. Below: when ranking is appropriate, how to analyze it, and what its limits are.

Ranking yields priorities: not just "important," but "what matters most." But the load on the respondent is higher - they have to think about the order. A list that's too long (8-10 items) overwhelms. The sweet spot is 4-6 options.

Order matters: a rank instead of multiple choice

Ranking - a question type in which the respondent orders options by a given criterion (importance, preference, frequency, and so on). Each option receives a rank - 1, 2, 3, etc. The result is an ordinal variable. The difference from multiple choice: there, order isn't recorded. Ranking reveals priorities but demands more cognitive effort from the respondent. Analysis - average ranks, "top-3" shares, competition between options.

In short: "put them in order" instead of "check the ones you want."

Drag-and-drop vs manual numbering

Drag-and-drop - the respondent drags items into the desired order. Intuitive, visual. On mobile, dragging can be awkward - small elements, a finger covering them. Numbering - type 1, 2, 3 into fields next to each item. Slower, but works everywhere. The choice depends on the platform and the audience.

In SurveyNinja the "Ranking" element supports drag-and-drop; on devices without drag support you can use an alternative input. Check it in the preview on mobile.

When ranking is appropriate

Prioritization: what matters most in a purchase, which criteria to pick for evaluation, in what order to consider things. Choosing from a set while accounting for order - that's ranking. Conjoint and MaxDiff are more complex methods; plain ranking is a quick way to get priorities.

When you don't need it: if order doesn't matter, multiple choice is enough. "Check all that apply" is easier for the respondent. Ranking makes sense when you specifically need a hierarchy.

Analysis: average ranks, top-3, preference matrix

The average rank for each option is the basic metric. The lower the average rank, the more important the option. An option with an average rank of 1.5 lands in first place more often. "Top-3" shares - how many respondents placed an option in the top three. Handy for reports: "price is in the top 3 for 85%."

A preference matrix - how many times option A ranked above B. It shows the competition between pairs. For simple tasks, average ranks and top-3 shares are enough.

In SurveyNinja: the "Ranking" element, reports, export

In the SurveyNinja builder there's a "Ranking" element. You add options and configure the input method (drag-and-drop or another). The respondent puts them in order. More detail - in the element settings.

Responses in the reports - the distribution of ranks for each option, the average rank. Excel export - a column with the order (for example "price, quality, delivery") or separate columns with ranks. For cross-tabulation and segmentation - average rank by group: do the priorities of men and women, of younger and older people, differ.

4-6 options - no more

Ranking 10 items is overload. The respondent gets tired and starts answering at random or by a pattern (first - 1, second - 2, and so on). 4-6 options is a reasonable limit. More - break them into groups or use another method (for example, paired comparison, MaxDiff).

Randomizing the order of options reduces the primacy effect - otherwise the ones first in the list tend to get high ranks. Turn it on in the settings if the display order carries no meaning.

Ranking vs rating importance on a scale

An alternative is to rate each item on a 1-5 scale ("how important"). Every aspect gets its own score. Plus: easier for the respondent. Minus: everything can get a 5 - priorities won't surface. Ranking forces a distinction: which matters more. But it only gives order, not intensity. "Price matters more than quality" - yes; "how much more" - no. The choice depends on the task.

Common mistakes

A list that's too long. 8-10 items - people give up or answer perfunctorily. Cut it down to 5-6 key ones.

Mixing criteria. "Rank by importance: price, quality, delivery, brand, sustainability" - if "sustainability" isn't relevant for some respondents, they'll put it last by default. Make sure all the options are comparable in context.

Forcing everyone to be ranked. Sometimes the respondent doesn't know or doesn't want to put things in order. A "not sure" option or the ability to skip reduces frustration but complicates analysis. Decide based on the task.

Partial ranking: top-3 instead of everything

One option is to ask for just the top-3 (or top-5) instead of a full ordering. Less load. But analysis is harder: options outside the top don't get a rank. For "what matters most," top-3 is enough; for a full hierarchy you need full ranking. Choose by the task.

Case: priorities when choosing a course

A survey of applicants: "Rank by importance: price, location, program, instructors, reviews." 5 items, drag-and-drop. 200 responses. Average ranks: price 2.1, program 2.3, instructors 2.8, reviews 3.5, location 3.8. Price and program are in the top 2 for the majority. Cross-tabulation by age: younger people rank price higher, those over 30 rank the program. Ranking gave priorities and the difference across segments.

Connection to MaxDiff and conjoint

For a more precise measurement of priorities, people use MaxDiff (choosing the best and the worst from a set) or conjoint analysis. Ranking is simpler but cruder. For quick surveys and initial prioritization, ranking is enough. For product decisions and pricing - conjoint and MaxDiff give more reliable estimates.

Ranking - ordering options by a criterion (importance, preference). Each one gets a rank. Analysis - average ranks, top-3 shares. In SurveyNinja - the "Ranking" element with drag-and-drop. Sweet spot 4-6 options.