A Beginner's Guide to Usability Testing

By Niklas Nordlof on Tuesday, May 06, 2014

A Beginner's Guide to Usability Testing

Usability tests are a powerful tool in shaping the design and development of a great site. Too often design is dictated by features that are cool or pretty, rather than the user experience. While you want a good-looking site, always remember that your visitors have a goal (probably several different goals) when visiting your site. Your job is to make it easy and pleasant for visitors to accomplish their goals. This is where usability testing comes in.

What's usability testing?

In case you're new to the term, usability testing is where you ask real, live people to use your product with various use cases in mind. The results of these tests then influence your design. So instead of just releasing your product (or site in the case of this article) into the wild and crossing your fingers, ideally you can catch problems before launch.

Sound good? Let's get started with these 5 easy steps.

Step 1: Decide when to test

Image courtesy of BBC.

Deciding when to test should be your first step. Often usability testing is an afterthought—especially in a small business scenario where you don't necessarily have a dedicated user researcher. 

Instead, you can schedule testing into the design process, so that results are able to really shape your design. And while any usability testing can have a positive impact, it's most effective when done several times throughout your design process.

Here are some examples of when you can test:

  • Before you design. Before designing your site, gather requirements by interviewing possible users (i.e., people who fit the demographics that you're targeting). Then use these requirements to build your first prototype. 
  • After each prototype stage. Depending on your design process, you might have several different prototyping stages. Since your design will already likely change at each stage based on the assumptions you make, test these assumptions. This way you can move ahead with development confidently, instead of wasting effort on features that hurt the user experience.
  • Before launch. Ready to launch? Finding any issues now will be much better than having angry customers tell you about them after your site is live.
  • Continuously after launch. Don't assume that your users will remain static. Your site will probably get new users all the time. Also, the desires and thought processes of your current users are not set in stone, which means your site shouldn't be.

Step 2: Test with use cases


Of course you should test everything, but, surprisingly, you shouldn't focus on specific features. Instead, focus on use cases. That's usability testing jargon for stuff your users would do. So, as you can see, use cases are task oriented.

Usability testing isn't about asking your users to design your site, it's about figuring out where users get stuck and what aspects offer a poor experience. All that is best determined by actually trying to use the product, which is what tasks are intended to simulate.

For example, if you have an ecommerce site, common tasks might be:

  • Finding a specific product. 
  • Browsing and filtering a list of products.
  • Buying a product. 
  • Returning a product.
  • Contacting customer support.

However, while use cases should be task oriented, they should also have a believable story (or scenario) surrounding the task. A scenario allows your test subjects to move from an artificial testing-mindset to a natural user-mindset. This means users will try to do things you hadn't thought of—like asking how to ship a product as a gift: without a price and to an address that's different than their billing address.

Here's an example scenario for finding a specific product:

"A friend's birthday is coming up. In a recent conversation, your friend mentioned that she was fermenting her own kombucha tea. You've decided to buy her Wild Fermentation by Sandor Katz. Find the book."

For browsing a list of products, the scenario might be: 

"A friend's birthday is coming up. In a recent conversation, your friend mentioned that she was fermenting her own kombucha tea. You've decided to buy her a book on home fermentation. Pick out a book for her."

So sit down and figure what you expect users to do on your site, and then write out some scenarios. 

Step 3: Recruit subjects


Who are your target users? Recruit representative subjects. If you're targeting software developers, then your buddy the barista might not be the ideal test subject. However, if you're developing an ecommerce site specializing in coffee, then your barista pal would make a great tester.

So that's the type of user that you're looking for, but the tough question is how do you find and convince people to participate. Unfortunately, there's not usually a simple answer to this question, but here are a few common tactics:

  • Offer rewards. While people will help you because they think the prospect of a usability test sounds interesting, you should also offer incentives. The most common method is to give people free products. Microsoft typically offers free software. A game studio might offer in-game items. If that doesn't fit your business model, you can also offer up lunch.
  • Ask everyone you know for help. While well-established companies have databases of people anxious to test, you might not have it so easy. This means that you're going to have to ask your friends, family, and co-workers for help. Even if they aren't the ideal test subjects, they might know someone who is. Send out company-wide emails. Post your request on Facebook, Twitter, and your personal blog.
  • Recruit from current users. If you already have a launched product, then ask those users to help test the new version. You can use emails, pop-up surveys, or post a request on your company blog.
  • Post in online communities. Depending on your site, posting a request for usability testers in a related online-community might be a great way to recruit users. 

Step 4: Set up your lab


High-end usability labs might have eye-tracking technology, multiple cameras for capturing different angles, audio recording equipment, partitions for additional observers, several different mobile devices, and so on.

However, depending on the type of testing you're doing, a conference room with a computer will probably be enough. With a basic usability test of a website, in general, you'll need:

  • Computer. So the user can access the site.
  • Screen recording software. Quicktime works well for this. Recording the screen means that you can go back over your user's actions after the test. You can also show these actions to any decision makers involved, which will help make your point that a feature needs to change.
  • Camera pointed at the user (optional). Your computer's built-in webcam works well for this, as it's non-invasive compared to a separate camera on a tripod pointed at the user. Putting this video and the screen capture side-by-side gives you a surprising amount of additional information about the user's emotions when using your site—it's easier to tell if a user feels frustrated or lost.
  • Audio recording. We'll talk about this more in the next section, but generally during a usability test you ask users to follow what's called think aloud protocol. This means that the user should speak out loud what they're thinking and doing. This is a lot of information that you can easily miss, so record it instead of relying on notes alone.
  • Note taking apparatus. In other words, a pen and notepad. Or a laptop if you prefer. Even if you're recording everything, taking notes now will help you pull out concrete findings later on.
  • A door. Basically, you want to limit distractions. If you have an open floor plan in your office, don't try to hold the test at your desk—reserve a conference room. Also, try not to have more than one person in the room, as you want your subject to feel at ease.

Step 5: Run the test


Image courtesy of Facit Digital.

Okay, you've got users, use cases, and a usability lab. Now it's time to test. Here are some tips for test day.

Before the test

Get ready before your subjects arrive. Here's what you should plan for and set up:

  • Keep it short. This is your job, so you're probably scheduled to be there for 8 hours, but your test subject has other stuff to do. So respect your subject's time by keeping the test short. A good length is 15 – 20 minutes, and you shouldn't go longer than an hour.
  • Schedule enough time. While you want to keep the test short, you also don't want to schedule too little time. Schedule the appropriate amount of time or your subject might have to leave in the middle of your test.
  • Schedule time between subjects. Even if you try to schedule enough time for an appointment, you might still go over. You don't want to cut into time scheduled for another appointment, so give yourself 30 minutes to an hour between appointments—that way you also have time to get things set up again.
  • Start the recording software. Start the recorder and get everything set up before your subject arrives. You have a limited amount of time with your subjects and you want to spend it testing—not setting up.

Once your subjects arrive

Be friendly and set the tone for the test. 

  • Offer beverages. It seems like a small thing, but any usability professional will tell you that this is part of their process. Offering subjects water, coffee, etc. is a good way to make sure that they're comfortable. 
  • Put your subjects at ease. Make sure that your subjects don't feel like they're taking a test. If your subjects feel uncomfortable or stressed out, you might not get accurate results. Try to create a relaxed atmosphere by talking and joking. Also limit the amount of people in the room to just you and subject so your subjects don't feel outnumbered.

During the test

Running a test is simple once you get the hang of it. Here are some common techniques:

  • Think aloud protocol. Usability guru Jakob Nielsen calls thinking aloud the #1 usability tool. Basically, you just ask your subjects to tell you what they're thinking. Have subjects talk about what they're looking at, what they're thinking, what they're feeling, and what they're doing. Along with providing precise feedback, this also lets you, as a facilitator, know when your subject is completely stuck and needs a little prompting.
  • Take notes. As you notice issues, write them down. The benefits of note taking are certainly well known, so just do it.
  • Don't tell your subjects what to do. The test isn't about walking your subjects through the site, it's about watching them use the site. So tell subjects scenarios and record what happens. There are some situations where your subject might get obviously stuck and frustrated (you'll know when these situations happen thanks to the think aloud protocol), and then it's okay to prompt them with the next step.

As with anything, after you run a few tests you'll learn what works best for you and for your site. Start integrating usability testing into your design process and see how people really use your site.

DevKit Templates

Find the website template you want in the Moboom Market