Usability testing

A user hunts for the "Place order" button for three minutes, clicks the wrong spot, and mutters "where is it?". Usability testing is a method for evaluating an interface by observing real users as they complete tasks.

It is not a survey asking "is it convenient?" - it is "watch what people actually do". It surfaces concrete problems: confusing navigation, muddled forms, non-obvious actions. The results are qualitative insights and metrics (task success rate, time on task). It connects to the customer experience (CX) and to qualitative research. With SurveyNinja you can run surveys after a usability session (SEQ, SUS) or collect feedback in a remote, asynchronous format. More on UX in research below.

A usability test is not a focus group. You do not ask "do you like the design?" - you hand over a task and watch whether people can do it.

Definition

Usability testing is a method for evaluating how usable an interface is by observing users as they perform typical tasks. Participants get scenarios ("place an order", "find the article about shipping") and the researcher records: did they complete the task, where did they stumble, how long it took, what they said. The goal is to surface usability problems before a wide release. Metrics: task success rate (the share who completed), time on task, number of errors. It is complemented by surveys: SUS, SEQ. It connects to feedback - the qualitative comments from participants.

In short: "user + task + observation" - you watch where the scenario breaks down.

Moderated and unmoderated testing

Moderated. The researcher is present (in person or over video): they hand out tasks, ask clarifying questions, and prompt the participant to "think aloud". It goes deeper - you can see hesitation and facial expressions and dig into problems. It costs more time.

Unmoderated (asynchronous). The participant completes the tasks on their own while a screen and voice recording is saved. It scales - you can launch 50 tests in parallel. There is less context - no chance to ask "why did you click here?". It suits simple tasks and quick feedback.

The choice depends on the tasks: a complex scenario or a prototype is more often moderated. A simple screen check or a large sample is unmoderated.

How many participants you need

Jakob Nielsen: 5 users surface 85% of the problems. In practice, 5-8 per iteration is enough to find the main pain points. More than that hits diminishing returns - new insights are rare. If the segments differ (novices vs experts, mobile vs desktop), use 5 per segment. An iterative approach - test then refine then test - is more effective than one "big" test with 30 people.

How to run a usability test

Tasks. Concrete scenarios: "Find the shipping cost to Moscow", "Place an order for $20", "Cancel the subscription". Not "look at the site" - an action with a clear success criterion.

Think aloud. Ask the participant to voice their thoughts: "I am looking for the pay button... I do not see it... maybe here?" - you hear the logic and the doubts. Do not prompt until they finish the task or get stuck.

Recording. Note down: completed/not completed, time, where they went wrong, quotes. A post-task scale - SEQ "How easy was it?" - gives a numeric snapshot. A post-session questionnaire - SUS, open-ended questions - complements the observation.

Respondents. Real users or a group close to your target audience. Not colleagues - they know the product. Convenience sampling is acceptable for usability: depth matters more than representativeness.

Usability test metrics

Task success rate. The share of participants who completed the task. 8 out of 10 is 80%. A low rate is a problem in the scenario.

Time on task. How many seconds/minutes to completion. Compare it against a benchmark or between interface versions.

Errors. The number of wrong clicks, dead ends, and failures before success. A qualitative breakdown shows exactly where people went wrong.

SEQ, SUS. A subjective rating after a task or a session. Combine it with objective metrics: "did it fast but rated it hard" is a signal of a hidden problem.

Connection to UX and surveys

User experience (UX) is a broad concept, and a usability test is one of the tools for measuring it. The test gives you "what is wrong" and "why", while surveys (NPS, SUS) give you "how bad" at scale. Combine them: a test with 8 people surfaces the problems, a survey with 200 confirms how widespread they are. Personas help you recruit relevant participants.

Common mistakes

Giving hints. "The button is at the top right" - this kills the value of the test. Wait for the participant to try on their own. Hint only if they are stuck for more than 2-3 minutes - and record that as a problem.

Complex or vague tasks. "Rate how convenient the site is" is not a task. "Find how to make a return" is a task with a clear success.

Testing on colleagues. They know the structure of the product. The results do not reflect real users.

Ignoring the mobile version. UX on phone and desktop is different. Use separate tests or separate tasks per device.

Post-session surveys in SurveyNinja

After each session - a short survey: SEQ for each task, SUS at the end, an open-ended "What was unclear?". In SurveyNinja, create a survey with logic: one block for task 1, another for task 2. You send the link to the participant after the session or have them fill it out right away. The export is a summary of subjective ratings. For unmoderated tests: the survey is embedded in the platform (if it supports it) or a separate link after completion. Hidden variables - the prototype version, the participant segment - let you compare cohorts.

When to run a usability test

Before a release - to check critical scenarios. After a redesign - to compare "before/after". When task success rate or conversion drops - to find the causes qualitatively. When adding a new feature - to make sure it is discoverable. Regularly (once a quarter) - to monitor the usability of key paths. It does not replace surveys - the test gives depth, surveys give scale.

In-person vs remote testing

In person. The participant is in an office or a lab. Full control, you see the nonverbal cues. More expensive, smaller geographic reach.

Remote moderated. Zoom, Teams, screen recording. Wider geography, cheaper. Connection quality affects how it goes.

Remote unmoderated. Platforms like Maze or UserTesting. The participant performs the tasks themselves and the system records. Scale, but less depth. A follow-up survey is a way to collect the subjective rating.

Case: usability test of a sign-up form

A subscription service. Sign-up was 4 steps, 12 fields. Analytics: 70% drop off at step 2. Usability test: 8 participants, task "sign up and start a subscription". Result: 5 of 8 completed it, 3 got stuck on the "organization type" field - they did not understand what to choose. Two left at step 2 because of the mandatory phone field - "I do not want to give it". The fix: the "organization type" dropdown was replaced with a hint, and the phone field was made optional. The retest - 8 of 8. A month later the sign-up success rate in production rose from 30% to 45%. The test pointed to specific fields, not just "the form is long" in general.

Usability testing is the observation of users as they perform tasks. It surfaces interface problems and provides task success rate and timing metrics. 5-8 participants are enough for one iteration. It connects to UX and to SUS/SEQ surveys. In SurveyNinja - post-session surveys for subjective ratings.