Join Chris Nodder for an in-depth discussion in this video How many people should you study?, part of UX Foundations: Usability Testing.
- The great thing about usability testing is that after running sessions with five participants you'll have seen about 80% of the issues that exist in your product. You can run more people but the benefits, the number of extra issues you find, drops off quite quickly. Also, even after five people it's likely that you'll have seen enough severe issues to keep you busy fixing things for a while. It's better to run a small study, make some changes, and then test again, to confirm your changes have the right affect, than it is to run one big study and just get more confident that you found all the issues.
Remember, you aren't looking for any kind of statistical significance. Instead, you're looking to identify and fix barriers to people using your product. If even just three of your study participants have an issue with part of your UI, it's worth investigating solutions because that indicates that many more people will have that same issue in the real world. It doesn't matter exactly how many people, the issue is likely to be severe enough that it needs fixing anyway. If you're used to recruiting for surveys, or rather, quantitive studies, you may be temped to run a whole bunch more people through your study.
Remember though, even if large numbers may give you confidence intervals and predict the exact number of customers who'd have that problem in real life, in this situation, we really don't care how many people have the problem. You just want to know it exists, and what triggers it so that you can fix it. Five people is enough to know it wasn't completely due to some random act that one individual under took. If three or more people in the study had the same issue, then it really merits extra attention. Because you'll be running several studies during the course of your product development process, you'll be gathering more and more data about users and the issues they have.
So although you might only run five participants in each study, you'll end it with 20 or 30 data points by the time you ship the product.
- What is usability testing?
- Finding the right participants
- Making a screener
- Asking the right questions
- Avoiding bias
- Making a task list
- Creating the test environment
- Running a pilot study
- Moderating sessions
- Capturing real-time observations
- Analyzing and reporting your results