How many responses you need

Tells you whether what you have is enough and how many more responses you need before conclusions hold

Free agent skill for Claude Code, Codex, Cursor and Gemini CLI. It works through the SurveyNinja MCP server, with your permissions.

Category
Understand results
Tags
sample, data quality
Works with
Claude Code, Codex, Cursor, Gemini CLI
MCP tools
3, 1 of them change data
Version
1.1, October 20, 2026
Made by
SurveyNinja
Price
Free
See also
Sample size calculatorMargin of error calculator
Skill file
SKILL.md

What it does

  • You know when to stop. “Can we draw conclusions yet?” and “how much longer?” are settled in one conversation.
  • Counts by group, not only in total. A breakdown by city or segment multiplies the requirement: each group needs its own volume.
  • No needless rigour. A survey for a quick internal decision does not have to be a research study, so you are not asked to collect more than you need.

Example

You type

Can we draw conclusions yet or should we keep collecting?

What the assistant brings back

318
Collectedcompleted
enough
Overall pictureabout 100 needed
±5.5%
Precision now
+80
For ±5%about 400 in total
  1. London 142 responses, enough
  2. Manchester 96 responses, almost there
  3. Leeds 44 responses, too few
  4. Other cities 36 responses combined, too few
National conclusions hold, city ones do not yet

The most common situation. The overall picture can be shown today. Comparing Leeds with London on 44 responses cannot: the gap will swing with every dozen new responses.

The numbers are an example. The estimate is made for your survey and your segments.

How it works

  1. Asks about the decision, not the percentageGetting a feel for the mood and choosing between two options need different precision, and that sets the sample size.
  2. Counts by segment, not in totalSplitting by city or segment multiplies the requirement: every group needs its own volume, and one overall figure is not enough.
  3. Gives honest rules of thumb30–50 responses for the general mood, around 400 for five-percent precision. These are conversation benchmarks, and the assistant calls them that.
  4. Can say “that is enough”When there is enough data, the assistant says so instead of advising you to keep collecting just in case.

Try also

  • How many responses do I need to compare two cities?
  • Is there enough data to make a decision from this survey?

What it will not do

  • Do not demand representativeness
  • Do not stay silent about a small sample
  • Do not count abandoned responses
  • Do not suggest a plan upgrade or lead to payment

The skill works with your permissions. Tools that change data are marked, so you check what the assistant is about to do before you confirm. To cut access, remove the connector in the assistant or delete the API key.

In short

What will the skill ask me?

Three things: how large the audience is, what you will decide from the results, and whether you need conclusions per group, such as per city or segment.

How many responses do I need?

About 30–50 to get a sense of the mood, around 100 to state shares to about 10%, and roughly 400 for about 5%. To compare two groups, 100 or more in each; for five segments, 100 per segment, so 500 or more.

Is this an exact calculation?

No, these are benchmarks for the conversation. If you need a formal estimate, the assistant says the numbers are approximate, explains what they depend on and points you to the free sample size and margin of error calculators on the SurveyNinja site.

What if there is enough overall but not per group?

That is the most common case. The assistant tells you which conclusions already hold and which could still reverse.

Are incomplete responses counted?

No, only completed responses count toward what has been collected. The skill reads the survey summary and report and changes nothing.

What do I need to use it?

A SurveyNinja account and an assistant that supports MCP and loads skills from a folder, such as Claude Code, Codex, Cursor or Gemini CLI. The assistant connects with your own API key or through “Allow access”. The skill is a plain text file and runs nothing by itself.

The full skill

The SKILL.md file the assistant reads, exactly as it is in the archive.

Show the full text

How many responses you need

Two questions come up constantly: "can we draw conclusions yet?" and "how much longer?". Both are settled in one conversation.

Reply to the person in the language they write in.

What to ask

  1. How large the audience is that the conclusion is about: a hundred customers or a hundred thousand. A small audience needs a small sample.
  2. What decision the results serve. "Get a feel for the mood" and "choose between two options" need different precision.
  3. Whether conclusions are needed per group. Breaking down by city or segment multiplies the requirement.

Estimate

get_quiz_summary — how many completed responses there are in total. It has no breakdown by group. For counts per city or segment, take the question that defines the group in get_quiz_report: it shows how many chose each option, over completed responses only. A slice by that answer is generate_filtered_report (not on every plan). If no question defines the group, ask the person how many fall into each.

Practical anchors for the conversation:

Goal Order of responses needed
Get a general sense 30–50
State shares to about 10% ~100
Precision around 5% ~400
Compare two groups 100+ in each
Break down by five segments 100 per segment, so 500+

These are conversation anchors, not a formal calculation. If a formal estimate is needed, say the numbers are approximate and what they depend on, and point to the free calculators on the SurveyNinja site: https://surveyninja.io/tools/sample-size-calculator, https://surveyninja.io/tools/margin-of-error-calculator and, for two groups, https://surveyninja.io/tools/min-sample-compare-calculator.

How to answer

  • Enough collected: say it plainly, collection can stop.
  • Not enough: say how much more is needed and warn that conclusions may reverse.
  • Enough overall but not per breakdown: the most common case. State which conclusions are already sound and which are not.

What not to do

  • Do not demand representativeness where a quick internal decision is being made.
  • Do not stay silent about a small sample when plans are being built on it.
  • Do not count abandoned responses toward the collected volume.
  • Do not suggest a plan upgrade or lead to payment. If a limit is hit, state the fact and stop.
The SurveyNinja ninja on a rocket

Connect SurveyNinja to your assistant

The MCP server works on every plan, Free included. Sign up, allow access and ask your assistant to build the first survey.