Concept testing: monadic, sequential monadic and comparative

Topic
Product and UX research
Contents

Concept testing is research that shows a product idea, ad, package, name or feature to people from the target audience before launch and measures how appealing, clear and believable it is and whether they would buy it. It helps you choose between ideas, fix weak points and drop concepts that will not sell before you spend money building them.

A concept is usually a short description with an image: what the product is, who it is for, what problem it solves and, if relevant, the price. Respondents rate it on a few standard measures and explain what they like and dislike.

Test designs

DesignWhat each respondent seesStrengthsWeaknesses
MonadicOne conceptCleanest read, no order or contrast effects, close to a real purchase momentNeeds a separate sample cell for each concept
Sequential monadicTwo or more concepts, one at a time, in rotated orderFewer respondents, each person compares several ideasOrder and contrast effects; longer survey
ComparativeConcepts side by side, then a choice or rankingClear preference between optionsShows which is preferred, not whether any of them is good enough
Proto-monadicSequential monadic ratings followed by a direct comparisonAbsolute ratings and a preference in one surveyInherits the order effects of sequential monadic

For a deeper comparison of the two most common designs, see sequential monadic testing and the guide monadic or sequential monadic.

What to measure

  • Purchase intent on a 5-point scale from "definitely would not buy" to "definitely would buy". The usual summary is the top-2-box share (probably plus definitely).
  • Overall appeal or liking.
  • Uniqueness: how different it is from what is available now.
  • Relevance: whether it solves a problem the person has.
  • Believability of the main claim.
  • Value for money if a price is shown.
  • Open questions on what they like most and least, which explain the scores.

Keep the same questions, scales and order for every concept so the results are comparable.

Sample size and reading the results

In a monadic test you compare independent groups, so plan the cell size from the smallest difference that matters. To detect a 10-point gap in top-2-box (40% against 50%) at α = 0.05 with 80% power, you need about 388 respondents per concept; a 20-point gap needs about 97. The two-group sample size calculator gives these numbers for your own inputs.

When the data is in:

  1. Compare top-2-box purchase intent between concepts and test the difference, for example with the A/B significance calculator.
  2. Look at the diagnostic measures to understand why a concept wins: is it more relevant, more unique or simply more believable?
  3. Read the open answers for fixable problems such as a confusing name or a missing feature.
  4. In a sequential monadic test, also compare each concept's scores when it was shown first. That subset behaves like a monadic test and shows how strong the order effect is.

Stated intent does not turn into sales one to one; Morwitz, Steckel and Gupta (International Journal of Forecasting, 2007) review how the two relate. Use the scores to rank concepts and to compare with your own past tests, not as a sales forecast.

Common mistakes

  • Concepts at different levels of finish. A polished render beats a rough sketch for reasons that have nothing to do with the idea. Present all concepts in the same format and length.
  • Selling instead of describing. Superlatives in the concept text inflate every score. Write plain, neutral descriptions.
  • No rotation. In a sequential design, the first concept is rated differently from the later ones. Rotate the order across respondents.
  • Too many concepts per person. Fatigue and contrast grow with each extra concept. Split a long list into several cells.
  • Wrong audience. Ratings from people outside the target market mislead. Screen respondents first.
  • Reading small gaps as wins. A 3-point difference with 150 people per cell is within noise.

Running a concept test with a survey

In SurveyNinja, add each concept as an image or text and write the same questions for every concept. Random order in SurveyNinja shuffles individual questions (the random sets mode shuffles questions only within a set), and it is available on every plan. For a sequential monadic test where each concept gets one question, put those questions in random order so the concepts rotate. If each concept has several questions, make one copy of the survey for each concept order and split the sample between the copies. For a monadic test, give each cell its own copy of the survey with its own link. Close each copy at its target with the response limit (Standard plan). Screening questions and screenout logic work on every plan, including Free. The A/B sample splitter calculates how many respondents each of two to four cells should get. Export the answers to CSV, XLSX or SPSS on any plan for the analysis.

Related methods: A/B testing compares versions with real behavior after launch, and conjoint analysis measures which features and prices drive choice.

0