Log in to like this post! You Don’t Need a Usability Lab Jim Ross / Wednesday, November 27, 2013 Sadly, a lot of usability testing doesn’t happen because people think it requires an expensive usability lab. The term “lab” itself implies some kind of high-tech science experiment. It sounds formal, expensive, time consuming, and out of reach. But you don’t need a lab, and usability testing is actually often better when it’s not conducted in a lab. Usability Testing in a Lab Sure, there are advantages to conducting usability testing in a lab, but there are also some significant disadvantages. Let’s take a look at these. Advantages of Testing in a Usability Lab Usability labs provide the best environment for people to observe and listen, either through one-way mirrors or through video cameras fed to large screens. More people come to observe usability testing when it’s conducted in a lab. Testing becomes more of an “event”. The novelty of it, the comfort of the observation room, the free food, and the chance to get out of the office are all powerful temptations to get people to attend. It’s always helpful to get people from the project team involved in observing the testing firsthand. When more people come to observe testing, you can have debriefing sessions and discussions at various points during the day. Usability labs give you the most control over the testing equipment, the environment, and the situation. You can ensure that each participant’s experience is the same. That’s important when you’re doing a test that relies primarily on collecting quantitative metrics. Labs allow you to have the most high-tech setup, with eyetracking, mobile usability testing equipment, multiple cameras, audio recording, etc. Since the participants come to you, you don’t have to travel, and you can fit more sessions in each day. Disadvantages of Testing in a Usability Lab A usability lab is a highly artificial environment. Taking people out of their normal context and bringing them into a lab does not show you their natural behavior. Usability labs with the one way mirror, the cameras, and the observers can be intimidating and make participants feel uncomfortable. This can affect their behavior. Knowing that the designers and project team are observing can lead to the effect of participants telling you what they think you want to hear. It’s harder to get people to participate in a usability test when they have to come to a lab. It’s easier to get them to participate when you can go to them or test them remotely. The best participants are often the ones that don’t have time to come to a usability lab. Those who are able to come to a lab are sometimes less-than-desirable or, even worse, “professional” participants who supplement their income by participating in focus groups and usability tests. Because people have to travel to the lab in person, your participants are limited to those in the immediate area. Usability labs are expensive. Lab space and equipment cost a lot of money, making usability testing more expensive. Alternatives to a Usability Lab If you don’t test in a usability lab, what are the alternatives? You can test participants remotely or you can go to the participants in person. Let’s look at the advantages and disadvantages of each method. Remote Usability Testing When you can’t meet with the participants in person, you can test them remotely by connecting with the participant through a phone call and a web conferencing application (like WebEx or GoToMeeting). The participant shares his or her screen so you can see them navigate through the interface. Project team members can observe by joining the web conference and listening in to the conference call. Advantages It’s easier to get people to participate in a 60 minute, remote session than it is to get them to come into a lab. Many participants are reluctant to leave work or home, but it’s easy for them to take an hour to participate remotely. You are more likely to get the right type of participants you need instead of “professional” focus group participants. You can reach participants anywhere in the world without having to travel. You can pay remote participants less because the test takes up less of their time. People participate on their own computer, in their own environment, so you may see more realistic behavior. Participants usually feel much more comfortable in their own environment, without a facilitator looking over their shoulder, without visible observers, and without cameras pointing at them. Their behavior is usually more natural when the awkward elements of the lab are absent. It’s often easier to facilitate the session when a participant isn’t in the room, because instead of focusing on maintaining eye contact and your interpersonal interaction with the participant, you can focus more on observing the screen and taking notes. It’s easier to get participants to think aloud as they perform tasks, because it seems to make sense to them that they need to explain what they’re doing since you’re not in the same room with them. Disadvantages Because you aren’t with the participants in person, you miss their facial expressions and body language. You miss the scowls, squinting, leaning into the monitor, head shakes, eye rolls, and yawns – all of which give you important signs of what the participant is feeling. You have to rely more on what they do onscreen and what they say. Sometimes you can observe them through a webcam, but that can be difficult for some people to set up. It’s more difficult to create a rapport with participants when you’re not in the same room with them. Participants sometimes have technical difficulties connecting to the web conference. These delays take valuable time away from the testing session. Although observers can connect to the web conference session and conference call, fewer people bother to observe remote sessions. Those that do tend to get distracted by email and other work. You miss out on the discussions that happen when everyone gathers in an observation room. Going to the Participants Another option is to go to the participants and do the testing where they would use the interface you’re testing. That’s often at their office or home. Advantages Visiting participants in their “natural” environment is by far the most realistic testing situation. You’re most likely to see how they would really use the interface you’re testing when they’re using their own equipment in their own location. Participants feel most comfortable doing the testing in their own location. Being with the participant in person is a little more personal than remote testing. You’re able to build a better rapport with the participant when you’re physically with them. It’s much easier to get people to participate when they don’t even have to leave their location. You don’t need to setup any special equipment, unless you want to record the session. It costs much less with no lab or equipment expenses. The testing is very easy and informal. Disadvantages You have less control over the situation when each participant uses their own equipment. So this is not the best form of testing when you’re gathering quantitative metrics and need each participant to experience the same conditions. Since they are using their own equipment, it’s more difficult to record the session. There are ways to record the session, but the quality may not be as good and it may add complexity to the session. You have to travel to the participants, which takes time and costs money. Unless the participants are all in the same area, travel time cuts down the number of sessions you can perform in a day. Going to the Participants and Setting up a Temporary Lab If all the participants are near each other, you can go to them, set up in one location and have them come to you. For example, if you’re testing employees at a company, you can set up in a conference room. Another example is going to a trade show or other event and setting up a “lab” in a booth or conference room. Advantages It’s easier to get people to participate when they are already in the area and your temporary “lab” is nearby. You can informally recruit people and get them to participate at the last minute. If you have no-show participants, you can easily get someone else to take their place. Because you’re in one location and don’t have to travel, you can fit more participants into each day. You have more control over the situation because you can set up the equipment and cameras ahead of time. It’s easier to record the situation with software you have installed on the computer and the cameras you can set up in the room. It’s easier when you need to use special testing equipment, like an eyetracker. Every participant uses the same equipment, resulting in more consistency. Although the participants are coming into this temporary “lab,” the location is usually less formal and intimidating than a usability lab. It’s less expensive than a usability lab. Disadvantages You don’t see people in their own environment, using their own technology. A temporary lab isn’t as intimidating or as unnatural as a traditional usability lab, but it’s not as natural and comfortable for participants as being at their own desk. You have to bring and setup a lot of equipment. So what Should You Use? Each of the techniques listed above are valid options. The best choice depends on the type of testing you’re doing, where your participants are, and how much money you have. There’s nothing wrong with testing in a usability lab. It’s the right choice for certain situations, but you shouldn’t think that a lab is necessary. The alternatives I’ve described are often more natural and produce better results. So if you don’t have (or can’t afford) a lab, don’t worry. It’s probably better anyway. Don’t let that stop you from testing. Image of observation room courtesy of Rosenfeld Media on Flickr under Creative Commons license.