Join Chris Nodder for an in-depth discussion in this video Asking the right questions, part of UX Foundations: Usability Testing.
- Asking your users direct questions doesn't always work very well. People are normally okay at answering questions that relate to things they've done in the past or tasks they perform regularly. These are called behavioral questions. On the other hand, people are not very good at answering questions that are forward-looking and speculative like, do you think you'd use this product? Or, how would you like to be able to do a certain thing? More to the point, they'll still give you an answer, but those answers aren't very believable.
There are a couple of reasons for this, one is that people's visualization of the thing you're talking about might be very different from your intentions. For instance, say you were talking about remote working. They may see a future with personal jetpacks to fly between meetings when instead you were thinking about teleconferencing. Another reason is that people just don't know what the future holds and what they say they'll do is often at odds with what they actually end up doing. For instance, most people would say they're very concerned about their online security, but then those same people end up reusing the same weak password on multiple websites.
Or people might tell you they intend to save money rather than spend it, but they still often give into short-term temptations. Why are we talking about this? Well, many types of research questions actually ask people to predict what they'll do in the future. Focus groups often show people screens and say, do you think you'd use this feature? Surveys ask people to say which of five potential features they'd find most useful. Market research interviews often ask people to visualize themselves using a product.
Because what people say and what they end up doing are often two different things the answers they give to these speculative questions aren't as trustworthy as we'd like. Also, people sometimes just tell white lies in order to impress. Think, for example, about how truthful the profiles on dating sites are. What we need to do instead and what usability tests are really good at is find questions that rely on people's real actions and demonstrated behaviors. Then, rather than asking people to predict what they'd do in the future we can watch them doing in right now.
The best way is to give people tasks to perform with our product and then see how well they can perform those tasks. That tells us whether we've chosen the right features to implement and whether we did it in a way that people can really use.
- What is usability testing?
- Finding the right participants
- Making a screener
- Asking the right questions
- Avoiding bias
- Making a task list
- Creating the test environment
- Running a pilot study
- Moderating sessions
- Capturing real-time observations
- Analyzing and reporting your results