How to Conduct a Usability Study
What is usability study?
A usability study is a user experience research method that evaluates how well representative users can perform tasks using a digital product. The process of conducting a usability study involves recruiting representative users, giving them realistic tasks to perform, and observing where and why they encounter difficulties.
If you’re new to conducting usability studies, we recommend you start with qualitative, moderated usability studies. These types of studies will be the focus of this article. Qualitative studies, as opposed to quantitative studies, gather non-numerical values and focus on “why” and “how.” Moderated usability studies, where the researcher is present during the session either digitally or physically, provide the flexibility to ask follow-up questions or modify the study.
Why conduct a usability study?
Proactive digital product teams conduct usability studies when they are building something new in order to validate their assumptions. By conducting a usability study throughout the design process, they can better understand how users currently use the product and ensure whatever they are creating provides value and is easy to use. When users are involved throughout the design process, teams are able to reduce waste—they aren’t developing and maintaining features and products people don’t want or can’t use—and are maximizing the digital product’s return on investment.
When teams don’t conduct usability studies as part of the design process, they often end up considering a usability study in reaction to negative feedback they receive post-launch. For example, users aren’t doing what they expected them to do or they are getting increased customer service requests.
How to design a usability study
Regardless of the reason prompting a usability study, it’s important to achieve a consensus on how the digital product relates to your business goals. That way, your team can focus you research questions and structure your study to ensure you are getting valuable information that will not only move the digital product forward, but align business goals with user goals—the key to getting a return on investment.
In addition to defining your business goals, decide what assumptions you need to validate. After looking at your website’s analytics, for example, your team may conclude “our users don’t do X because of Y.” Or after a flurry of customer service emails, your team may speculate that the mobile app’s navigation is confusing to users. These are assumptions that you need to validate.
By laying out your business goals and assumptions up front, you’re in a better position to develop your research questions. These questions will determine how you structure your study and, ultimately, the value you get from it. Your research questions will be unique to your goals and digital product, but examples include:
What roadblocks do users encounter when they add a new lesson?
How do we encourage new users to submit an application?
Your goal is to answer your research questions by the end of the study. Assuming you have about an hour with each participant, three to five research questions tend to be about all you can reasonably expect to accomplish in one usability study.
Your research questions will dictate what you’ll have participants do during the study. Task scenarios give participants something to do using the product with a short, but realistic back story. The wording of the task scenarios is important. Don’t give away the answer by using your product’s vocabulary or the items in your navigation menu.
For an application that calculates moving costs, a task scenario might be: You’re moving to a neighboring town on April 15. Using this app, determine what your moving costs will be.
Next, you’ll need to find representative users as participants. Studies have shown you can uncover the majority of usability issues with just five participants. There are circumstances where you will need more. For example, if one of your research questions involves understanding differences between types of users, you may need to recruit five people for each group.
Focus on finding people who have the same goals and exhibit the same types of behaviors as your actual users. Demographics tend to be less important than behaviors. You can find participants using your personal network, but avoid friends and family, who are less likely to tell you their true feelings. You can also recruit participants using social media, industry associations, and market research firms. If you offer an incentive, in the form of a gift card or cash, you are much more likely to get a response.
During the Usability Study
Conduct a dry run of your usability study a day or two before your actual study begins. This will allow you time to fix technical issues and reword task scenarios that don’t make sense.
When you begin a usability study session, remind the participant that you’re not testing them, instead you’re evaluating the digital product. Ask them to talk aloud about what they are doing and thinking, but expect to have to prompt them every so often throughout the session.
If you’re acting as the moderator, it helps to have someone take notes for you during the study. That way you can give the participant your full attention but still capture what is happening in real time. We find this useful even if we are recording the study. Ideally, your whole team can watch the usability study in another room or remotely through screensharing software. Make sure they take notes, as well. Their perspective will be important when you are prioritizing issues and brainstorming solutions.
Outside of moving the participant from task to task and prompting the participant to talk aloud, the moderator should be silent for most of the study, observing what is happening and where the participant is having trouble.
Developing Usability Insights and Recommendations
When the study is complete, the moderator should debrief with the notetaker and/or team and make sure they have captured as much information as possible. As soon as possible, compile this data in a spreadsheet to record the main issues encountered in each task scenario, by participant. Group these issues into categories and prioritize by:
How often they were encountered in the study.
How likely it is your users will encounter the same issues.
How likely those issues are to prevent users from completing important tasks.
The team can then brainstorm solutions to the most pressing usability issues. Some issues may be solved with a simple wording change. With fixes that are relatively simple and affect a large amount of users in performing critical tasks, it makes sense to prioritize the fix. Other issues, like rethinking a complicated user flow based on how users actually do things, may take longer but might be absolutely essential for the product’s—and the business’s—success. You can also use the data you gather to uncover opportunities to provide added value, fulfill unmet needs, and deliver an exceptional user experience.
Usability studies lead to more usability studies. Your solutions need to be validated and you should be conducting usability studies throughout the lifetime of your digital product. After your first usability study, you can use the data to benchmark your progress in subsequent studies. You may also uncover new user experience metrics that you can implement in your analytics tool or using other types of quantitative and qualitative research.
Is your team interested in conducting usability studies? We cover the topics in this article, as well as provide in-depth information about designing the study, recruiting users, and analyzing the results in our How to Conduct Usability Studies workshop.