How many responses you need
Tells you whether what you have is enough and how many more responses you need before conclusions hold
Free agent skill for Claude Code, Codex, Cursor and Gemini CLI. It works through the SurveyNinja MCP server, with your permissions.
- Category
- Understand results
- Tags
- sample, data quality
- Works with
- Claude Code, Codex, Cursor, Gemini CLI
- MCP tools
- 3, 1 of them change data
- Version
- 1.1, October 20, 2026
- Made by
- SurveyNinja
- Price
- Free
- Skill file
- SKILL.md
What it does
- You know when to stop. “Can we draw conclusions yet?” and “how much longer?” are settled in one conversation.
- Counts by group, not only in total. A breakdown by city or segment multiplies the requirement: each group needs its own volume.
- No needless rigour. A survey for a quick internal decision does not have to be a research study, so you are not asked to collect more than you need.
Example
You type
Can we draw conclusions yet or should we keep collecting?
What the assistant brings back
- 318
- Collectedcompleted
- enough
- Overall pictureabout 100 needed
- ±5.5%
- Precision now
- +80
- For ±5%about 400 in total
- London 142 responses, enough
- Manchester 96 responses, almost there
- Leeds 44 responses, too few
- Other cities 36 responses combined, too few
The most common situation. The overall picture can be shown today. Comparing Leeds with London on 44 responses cannot: the gap will swing with every dozen new responses.
The numbers are an example. The estimate is made for your survey and your segments.
How it works
- Asks about the decision, not the percentageGetting a feel for the mood and choosing between two options need different precision, and that sets the sample size.
- Counts by segment, not in totalSplitting by city or segment multiplies the requirement: every group needs its own volume, and one overall figure is not enough.
- Gives honest rules of thumb30–50 responses for the general mood, around 400 for five-percent precision. These are conversation benchmarks, and the assistant calls them that.
- Can say “that is enough”When there is enough data, the assistant says so instead of advising you to keep collecting just in case.
Try also
- How many responses do I need to compare two cities?
- Is there enough data to make a decision from this survey?
What it will not do
- Do not demand representativeness
- Do not stay silent about a small sample
- Do not count abandoned responses
- Do not suggest a plan upgrade or lead to payment
The skill works with your permissions. Tools that change data are marked, so you check what the assistant is about to do before you confirm. To cut access, remove the connector in the assistant or delete the API key.
In short
What will the skill ask me?
Three things: how large the audience is, what you will decide from the results, and whether you need conclusions per group, such as per city or segment.
How many responses do I need?
About 30–50 to get a sense of the mood, around 100 to state shares to about 10%, and roughly 400 for about 5%. To compare two groups, 100 or more in each; for five segments, 100 per segment, so 500 or more.
Is this an exact calculation?
No, these are benchmarks for the conversation. If you need a formal estimate, the assistant says the numbers are approximate, explains what they depend on and points you to the free sample size and margin of error calculators on the SurveyNinja site.
What if there is enough overall but not per group?
That is the most common case. The assistant tells you which conclusions already hold and which could still reverse.
Are incomplete responses counted?
No, only completed responses count toward what has been collected. The skill reads the survey summary and report and changes nothing.
What do I need to use it?
A SurveyNinja account and an assistant that supports MCP and loads skills from a folder, such as Claude Code, Codex, Cursor or Gemini CLI. The assistant connects with your own API key or through “Allow access”. The skill is a plain text file and runs nothing by itself.
The full skill
The SKILL.md file the assistant reads, exactly as it is in the archive.
Show the full text
How many responses you need
Two questions come up constantly: "can we draw conclusions yet?" and "how much longer?". Both are settled in one conversation.
Reply to the person in the language they write in.
What to ask
- How large the audience is that the conclusion is about: a hundred customers or a hundred thousand. A small audience needs a small sample.
- What decision the results serve. "Get a feel for the mood" and "choose between two options" need different precision.
- Whether conclusions are needed per group. Breaking down by city or segment multiplies the requirement.
Estimate
get_quiz_summary — how many completed responses there are in total. It has no
breakdown by group. For counts per city or segment, take the question that
defines the group in get_quiz_report: it shows how many chose each option,
over completed responses only. A slice by that answer is
generate_filtered_report (not on every plan). If no question defines the group,
ask the person how many fall into each.
Practical anchors for the conversation:
| Goal | Order of responses needed |
|---|---|
| Get a general sense | 30–50 |
| State shares to about 10% | ~100 |
| Precision around 5% | ~400 |
| Compare two groups | 100+ in each |
| Break down by five segments | 100 per segment, so 500+ |
These are conversation anchors, not a formal calculation. If a formal estimate is needed, say the numbers are approximate and what they depend on, and point to the free calculators on the SurveyNinja site: https://surveyninja.io/tools/sample-size-calculator, https://surveyninja.io/tools/margin-of-error-calculator and, for two groups, https://surveyninja.io/tools/min-sample-compare-calculator.
How to answer
- Enough collected: say it plainly, collection can stop.
- Not enough: say how much more is needed and warn that conclusions may reverse.
- Enough overall but not per breakdown: the most common case. State which conclusions are already sound and which are not.
What not to do
- Do not demand representativeness where a quick internal decision is being made.
- Do not stay silent about a small sample when plans are being built on it.
- Do not count abandoned responses toward the collected volume.
- Do not suggest a plan upgrade or lead to payment. If a limit is hit, state the fact and stop.
More skills
All skills- A/B variant analysis Tells you whether the difference between variants is real or just noise
- Survey results summary Answers "so what did the survey say" with figures and a conclusion instead of a raw export
- Comparing with past results Compares a survey with its own past, because one figure means little and a change says a lot
- Analyze open-ended answers Turns a hundred lines of free text into a handful of themes with numbers and quotes
- Completion funnel Shows which question makes people close the tab, and why
- Comparing traffic sources Shows which channel brought the right people and which only brought numbers
Connect SurveyNinja to your assistant
The MCP server works on every plan, Free included. Sign up, allow access and ask your assistant to build the first survey.
25 Questions for a Travel Agency Customer Satisfaction Survey