Doing small and simple usability studies usually give you a great return on investment - testing as little as 5 subjects can yield up to 85% of all usability problems on a site according to a study by Jakob Nielsen. So how do you prepare for your own web usability study?
A copy of the preparation checklist we used for our latest usability study.
Below is the preparation checklist we, here at Baymard Institute, went through before conducting each test in our latest web usability study. My comments are in [brackets]. If you want an easy printable version, then I’ve made a .PDF version you can use (without the comments).
Note: you should obviously add items as your study prescribes it, this is only meant as inspiration and/or a starting point.
Bring along
Don’t waste your own or the test subject’s time by forgetting necessary equipment so be sure to bring:
- A printed version of this checklist
- Pen and paper [For taking notes on test subjects behavior]
- Laptop (with required software installed) [To document the user behavior for later analysis we use a screen recording software that simultaneously record what’s happening on the computer screen as well as the subject’s face with the built-in webcam. We also use a mouse gaze heatmap tool we’ve developed ourselves. We will write posts on both of those tools in the near future.]
- Laptop charger
- Plenty available space on the laptop’s hard disk for the screen recording software. [Screen recording software will typically consume roughly 3GB per hour of recording]
- Computer mouse [So the subject won’t have to use the trackpad. We also give them the option of using their own mouse if they prefer.]
- Ethernet cable
- 3G mobile internet connection [We prefer to conduct each test in the subject’s private home whenever it’s possible, as this sets a more realistic setting for the subject’s level of confidence and privacy. Whenever we do this there is a chance their internet connection is/goes down during the test, and therefore we bring our own mobile internet as a backup.]
- Customized site sequence [If you’re testing multiple sites, the test sequence is very important, so you’ll want to expose each subject to a new sequence.]
- Fictional character (made-up personal information) [If personal information is be to submitted during the test then have a fictional character’s personal information printed on a piece of paper. You can include information such as name, gender, age, address, e-mail and so on. When we test we always give the subject the option between using our made-up data or their own]
Computer settings
Some browser settings need to be adjusted before conducting each test so the history of previous tests don’t interfere:
- Clear browser cookies
- Clear browser history
- Disable browser auto-filling of form fields
- Set browser’s homepage to the custom test-page (with customized site sequence) [This way we give the test subject an easy way back to the main test site. If you test multiple sites then create a very simple HTML page with links to each of your test pages.]
Brief
Give the subject a short brief of what the website usability test is about and reassure them you’re testing the sites and not the subject himself:
- “This is not a test of your computer skills so you can’t do anything wrong. This is a test on how user-friendly these websites are. Some sites may be designed badly which can make them difficult to use, however, this is exactly what we’re after, so the more of these design problems you uncover on each site the better you’re doing.” [This turns the subject’s “mistakes” into design problems they should be proud of uncovering.]
- “Please pretend you’re at home/work during this test and were to complete these tasks on your own. If there’s anything you wouldn’t normally do, then let us know and we’ll make note of it. We may ask you to finish what you’re doing anyways but it’s important for us to know if you wouldn’t have gone any further under normal circumstances.”
- “When a site asks you for personal information you can either use your own data or this fictional character we’ve created for you. (Hand over paper with personal info.)”
- “Everything you say and do will remain anonymous. The end result of this is not to profile you but to make websites more user-friendly” [Or whatever purpose you may have with the test you’re doing.]
What du you think of this way to do a research design? Do you have any other important things to remember?
Note: we’ve removed certain items from the above lists that were specific to the study we were conducting.
