UMUX: Product Usability Metric

UMUX (Usability Metric for User Experience) is a four-item questionnaire that measures the perceived usability of a product, app or website on a 0–100 scale. It was published by Kraig Finstad in 2010 as a shorter alternative to the 10-item System Usability Scale (SUS), and its scores are designed to be read on a similar scale. UMUX-Lite is a two-item version for places where even four questions are too many, such as in-app surveys.

Finstad built the four items around the ISO 9241-11 definition of usability: effectiveness, efficiency and satisfaction. That makes UMUX a quick way to track usability between releases, compare product versions or add a usability measure to a longer survey.

The four UMUX questions

Respondents rate their agreement on a 7-point scale from 1 = strongly disagree to 7 = strongly agree. Replace [this system] with the name of your product.

1. [This system's] capabilities meet my requirements.
2. Using [this system] is a frustrating experience.
3. [This system] is easy to use.
4. I have to spend too much time correcting things with [this system].

Items 1 and 3 are worded positively, items 2 and 4 negatively. The mix makes people read each statement, and it means the even items have to be reversed when you score.

How to calculate the UMUX score

1. Odd items (1 and 3): score − 1.
2. Even items (2 and 4): 7 − score.
3. Add the four values. The maximum is 24.
4. Divide by 24 and multiply by 100 to get the respondent's UMUX score.
5. Average the scores of all respondents.

Worked example

A user answers 6, 2, 7 and 3.

• Item 1: 6 − 1 = 5
• Item 2: 7 − 2 = 5
• Item 3: 7 − 1 = 6 (the answer was 7; odd items are score − 1)
• Item 4: 7 − 3 = 4

Total = 20. UMUX = 20 ÷ 24 × 100 = 83.3.

If you use a 5-point scale instead, subtract 1 from odd items, subtract even items from 5, and divide the total by 16. Keep the same scale in every wave, because 5-point and 7-point results are not directly comparable.

UMUX-Lite: the two-item version

UMUX-Lite was proposed by James Lewis, Brian Utesch and Deborah Maher in 2013. It keeps only the two positive items:

1. [This system's] capabilities meet my requirements.
2. [This system] is easy to use.

Scoring: subtract 1 from each answer, add the two values, divide by 12 and multiply by 100. Answers of 6 and 7 give (5 + 6) ÷ 12 × 100 = 91.7.

Lewis and colleagues found that raw UMUX-Lite scores did not line up exactly with SUS, so they proposed a regression adjustment to bring them onto the SUS scale: adjusted UMUX-Lite = 0.65 × UMUX-Lite + 22.9. In the example, 0.65 × 91.7 + 22.9 = 82.5. Use the adjusted score whenever you compare UMUX-Lite with SUS results or SUS benchmarks, and say which version you report.

How to interpret UMUX scores

UMUX was built to track SUS, so teams usually read it against SUS reference points. The most cited one comes from Jeff Sauro's analysis of SUS data (2011): an average SUS score of about 68. A UMUX score clearly above 68 suggests usability above that average, and a score well below it points to problems worth investigating.

• Compare like with like. The same item wording, scale and type of user from one release to the next.
• Look at the items, not only the total. A low score on item 4 (time spent correcting things) points to a different fix than a low score on item 1 (capabilities).
• Check the sample size. With 15 respondents the average can move several points by chance. Report a confidence interval when you compare versions.

UMUX vs SUS

If you need comparability with the largest body of published data, SUS is the safer choice. You can score SUS with the SUS calculator.

Common mistakes

• Forgetting to reverse items 2 and 4. The score then measures nothing useful.
• Rewording the items or changing the scale between waves.
• Comparing raw UMUX-Lite with SUS without the regression adjustment.
• Asking people who have barely used the product. UMUX measures experience, so ask after real use or after a task.

How to run UMUX in a survey

1. Choose UMUX or UMUX-Lite depending on how much space the survey has.
2. Add the items as a matrix or as separate scale questions with a 1–7 agreement scale and labeled ends.
3. Send it after meaningful use: at the end of a usability session, after onboarding or to active users.
4. Export the answers, apply the scoring steps per respondent, and report the mean score, the number of respondents and the item averages.