What Is Usability Testing?

Usability testing is the process of testing a website or app by watching people try to use it and observing their behavior and reactions. While there are many ways to conduct a usability study, usability testing involves directly observing real or representative users interacting with a website.

Each year, Baymard Institute conducts 20,000+ hours of new research to produce actionable insights for improving e-commerce UX. Usability testing is the foundation of our research, and it’s used by some of the world’s leading sites to identify usability issues and improve the user experience.

Let’s dig into why usability testing is important, and what types of testing are most effective for maximizing e-commerce conversion rates.

Why Is Usability Testing Important?

When performing website usability testing, the goal is to reveal areas of confusion and uncover opportunities to improve UX.

In-depth knowledge can blind designers and developers to a website's usability issues. Testing with people who have no familiarity with the site is important to reveal how the intended users experience it.

During most usability tests, participants will attempt to complete tasks while observers watch, listen, and take notes on what the users are doing. The observers will identify specific elements and processes causing participants to experience usability issues and roadblocks during the shopping and buying process. The observers will also identify when participants have a smooth experience.

The testing process reveals whether real people:

• Understand how a site works
• Become lost or confused using the site
• Can complete tasks successfully
• Encounter usability issues
• Can find key information
• Are satisfied with their overall experience

Usability testing is an important step in the UX design process, whether you’re designing new sites or improving existing designs. Without testing your site with real users, you may be stuck with a UX design that your team members understand, but your target audience does not.

Usability researchers collect qualitative and quantitative data, depending on the type of test and the goal of the testing process. Using the data, you’ll be able to identify the changes you need to make to improve user experience and satisfaction on your site, so you can increase conversion rates.

Usability Testing Methods

What are the best ways to get real people to interact with your website, so you can observe their behavior, evaluate the user experience, and identify usability issues? Here is a list of the top 5 usability testing methods.

1. Hallway Testing

In hallway testing, test participants are chosen at random from a workplace or a public place like a coffee shop or mall. They are asked to perform a quick usability test on your website or app. Researchers may offer a gift card or other incentive to encourage participation.

The only people not eligible for participation in hallway testing are the designers and developers who are actively involved in the project.

This type of testing is a good way to collect a large amount of qualitative data that can help validate design functionality, but it’s not good for extensive testing or follow-up processes.

2. One-to-One “Think Aloud” Testing

“Think Aloud” testing protocol involves participants performing a set of specified tasks while thinking out loud about the process. They are invited to say whatever comes to mind as they are completing the tasks, including what they’re seeing, thinking, feeling, and doing.

This kind of usability testing gives observers valuable insight into the cognitive processes happening in the users’ heads, while they can also observe users’ ability to complete a task, and how fast they were able to complete it.

UX testing can be recorded, and verbalizations can be transcribed and analyzed by researchers. Observers can also take notes as they interpret test participants’ actions and words, noting places where they encounter friction and seeing how they reacted to problems.

During Baymard’s “Think Aloud” usability studies, we task users with finding, evaluating, selecting, and purchasing products or services matching everyday purchasing preferences. We also give participants administrative tasks such as “try to change your account password”.

Participants are instructed to behave just as they would on their own, including abandoning a site and going to a competitor, or going off-site to search for information while shopping.

When they experience problems, the participants are asked open-ended questions such as:

• What are you thinking right now?
• Why did you click there?
• What did you expect would happen?

Because you can hear participants talk through what they’re thinking as they progress through your site, this type of testing can provide valuable information on reasons for cart abandonment.

3. Quantitative Studies

Quantitative usability testing can provide metrics for things like:

• The number of clicks required to complete a task
• Time users take on a task
• Success and failure rates
• Instances of confusion or abandonment
• Satisfaction scores
• Number of CAPTCHA errors

Since quantitative metrics are simply numbers, they can be difficult to interpret. For example, if a user leaves a product page, a quantitative study won’t reveal why. Is the description difficult to read? Are there not enough images for them to evaluate the product?

For that reason, it can be helpful to compare quantitative usability metrics with known standards or benchmarks, or with metrics from a previous design iteration.

4. Eye-Tracking Testing

The human memory is fallible, and it can be difficult for test participants to remember their behavior after the fact.

Eye-tracking technology makes it possible to measure the duration and location of a user’s viewing behavior. It records where someone is looking, what they’re looking at, and for how long.

When a user’s gaze stops moving, that area is called “fixation”. The area between fixation points is a “saccade”. By tracking saccades, we can see the path the eye is taking on the page. Usability testing with eye-tracking can provide valuable insight into user behavior, which can reveal UX issues and opportunities for improvement.

For instance, eye-tracking testing can reveal:

• How users scan your product pages
• What patterns users follow as they explore your pages
• Which page elements (including images and text) get the most attention
• If your checkout form is optimized for quick scanning, so users can understand all the required fields at a glance

All eye-tracking data is collected automatically, without disturbing the flow of the test. You can view the eye-tracking test live, so you can see what the user sees and immediately see usability problems.

5. UX Audit

During a UX audit, you bring in experts to evaluate your website. Evaluators typically measure effectiveness and efficiency based on usability principles like clarity, recognition, and credibility.

Baymard offers a comprehensive UX audit service for e-commerce sites that includes a full analysis based on our 71,000+ hours of UX research.

Our UX researchers will write a detailed 120+ page report outlining 40 prioritized and research-backed user experience improvements for your site, providing UX best practice implementation examples from leading sites.

Find out more about Baymard’s comprehensive audit services.

What Usability Testing Is Not

There are some methods that are commonly confused with usability testing. These UX tools can help you improve your customer experience, but they don’t qualify as usability testing because they don’t let you observe users' behavior directly while they’re using your site.

Here are some tools that many people mistakenly refer to as usability testing:

A/B Testing

A/B testing, often called split testing, is an experimental approach to UX design that aims to identify changes that maximize conversions.

During testing, you can compare two versions (A and B) that are identical except for one variation that may impact a user's behavior. For instance, you can test different versions of page titles or images on a product page.

While A/B testing is an effective way to experiment with changes to see how they impact the performance of your site, it’s not a substitute for usability testing.

Heatmaps

Heat mapping software creates a visual representation of how users move around by indicating the hottest (most popular) and coolest (least popular) parts of the page. With data from mouse movements, clicks, and scrolling, heatmaps create a colorful visualization of users’ attention.

Eye-tracking heatmaps can be a helpful tool for UX research, but without additional testing methods, they provide an incomplete picture of user experience.

Surveys

Although they can be used to conduct UX research or gauge customer satisfaction, surveys won’t replace usability testing. Surveys don’t allow researchers to observe visitors using the site and instead rely on users answering questions about a past experience.

Can You Do Your Own Usability Testing?

Your internal team, including UX writers, designers, and developers, probably test your website or app all the time — but that’s not true usability testing.

Anyone involved in the creation of a product is inherently biased and likely to have significant blind spots preventing them from noticing UX issues. Even employees who are not involved in designing the site will be too familiar with your products and services to give the kind of honest results that real users can.

While you may not have the resources or expertise to conduct usability testing, you can measure and improve the UX performance of your site against e-commerce best practices based on Baymard’s large-scale usability research.

With the tools in Baymard Premium, you can self-audit your e-commerce site, down to thousands of specific elements and page designs. Membership includes individual onboarding support from a UX researcher to help you identify the most impactful changes for your site.

Learn from Real-World E-Commerce Usability Testing Results

Baymard conducts thousands of hours of usability testing to discover the best and worst UX design patterns.

That testing includes 1,900+ qualitative moderated in-lab usability test sessions following the “Think Aloud” protocol — real users testing live e-commerce sites — combined with in-lab eye-tracking testing and quantitative studies with 14,453 users.

During testing of the desktop and mobile websites of more than 90 leading U.S. and European brands, test participants encountered over 14,500 usability issues.

We’ve analyzed these issues and created 550+ e-commerce user experience guidelines to help you improve your site’s UX performance and conversion rate.

E-commerce businesses of all sizes use our continually updated research to:

• Speed up and validate design decisions.
• Consistently develop high-performing e-commerce sites.
• Learn what it will take to outperform competitors’ UX.
• Avoid going down the wrong path early in the design and development process.
• Get UX inspiration from the world’s leading e-commerce sites.

Start putting our large-scale usability testing results to work on your site by accessing Baymard Premium.