6 Tools for Remote Unmoderated Usability Testing

Usability testing can take many forms & methods. The most common are :

In-Person Testing

During an in-person test, a user experience professional acts as the moderator and leads the session. The moderator either asks participants to complete specific tasks or observes participants as they complete tasks naturally and then asks follow-up questions.

In-person usability testing (one-on-one with a moderator) is the preferred method when:

• · In order to control for confounding variables, a laboratory setting is required.
• · Use of special equipment is needed.
• · The website being tested is behind a firewall or can only be accessed via a secure connection.
• · There is need to see the participant’s facial expressions and physical movements.
• · Screen sharing is not an option.

 

Remote Testing

Remote usability testing is a technique where user observation can be done with screen sharing applications. When usability evaluators and prospective users are located in different countries and time zones, conducting a traditional lab usability evaluation creates challenges both from the cost and logistical perspectives. These concerns led to research on remote usability evaluation, with the user and the evaluators separated over space and time.

Remote testing is most useful when:

• · The resources required for in-person testing make such testing prohibitive.
• · Participants are unable or unwilling to travel to the test site.

 

Types of Remote Testing :
While an in-person test session is, by definition, moderated, remote testing can be conducted either with or without a moderator.

• Moderated remote testing is typically conducted one-on-one with a moderator via Skype, Webex or other screen-sharing application. It allows us to collect quantitative and qualitative data about participant’s behavior similar to in-person testing. We are able to measure time-on-task, error rates, attention and navigational issues, etc. as well as participant frustration, confusion and dissatisfaction. While not as informative as in-person testing (since we can’t actually see them), participants are typically pretty good at talking about what’s bothering them.
• Unmoderated remote tests are automated and have the advantage of being “set it and forget it”, allowing data to be collected at the participant’s convenience. This lets UX folks do test sessions with hundreds of users simultaneously, in their natural environment. The self-reported feedback and comments which are received in response to open-ended questions can be the most valuable data you collect during an unmoderated test.Such automated testing typically allows us to collect quantitative data about participant behavior using a larger sample size than moderated testing methods and varying comparative data.

 

Automated Remote Testing Tools :
In recent years, a variety of remote testing tools have become available that are automated. Here is a list of my top 6 Usability Testing tools that I think are pretty cool in the way they perform and deliver results.

 

1.Chalkmark

Price
$109/month

How it Works
It’s possible to upload images/screenshots, set user tasks, and recruit participants with the help of a unique study link Chalkmark provides. Chalkmark presents participants with a task and a screenshot. Participants click links in the screenshot where they think they would find the information they need. Chalkmark proceeds through the remaining tasks and screens. The test results are obtained in the form of heat maps that show concentrations of clicks and how many participants clicked each area of a screen.

2.Loop 11

Price
$158 per month or $350 per project.

How it Works
Loop¹¹ uses live HTML pages rather than screenshots. An obvious benefit is that you don't have to go through the hassle of creating screenshots. The software allows you to create tasks and questions to help you explore the user experience. It has powerful reporting features like task completion rate, time per task, most common success page, most common fail page, most common first click, most common navigation path, detailed participant path analysis, and number of page views to complete tasks.

3.OpenHallway

Price
$49 - $199/month

How it Works
You can create a scenario with navigation instructions and a recording link that is sent to users. This supports screenshots and images of the product to be tested. OpenHallway records a participant’s screen and voice, and then uploads a video for you to watch after the test is completed. Test participant notes are shown below the video, and you can also add notes of your own.

4.Usabilla

Price

$49/month to $199/month

How it Works Usabilla shows screenshots to participants and asks them questions which they answer by clicking and/or annotating the image. You choose what questions to ask from a preset list, provide your own or both. You can provide the screenshots or just input the URL and Usabilla will take the screenshots for you. Usabilla hosts the test (it appears in an overlay) but does not recruit users for you. Instead, they provide a link to send to participants, or alternatively, a piece of JavaScript code to place on your site, which will invite users to take the test.

5.UserTesting

Price:
$39 per participant, per test.

How it Works:
You can select your participants, describe the tasks to be performed and test your website, competitors' websites, prototypes, Facebook games, Google ads, and mobile apps. The great thing about this is the quick feedback, in the form of videos of participants using the site/application, voice recordings, and the ability to interact with users after the test to ask follow-up questions. Sharing results is also made easy with multiple logins for team members.

6.UserZoom

Price
$1,000 per month.

How it Works:
UserZoom is a comprehensive suite of user-testing tools. It allows you to create remote usability (task-based) studies, card sorts, true-intent, voice of customer, tree tests, click tests, and more. UserZoom captures survey data, task metrics (success ratios, time on task, number of clicks, and number of unique pages viewed), participant behavior, and system information. Although on the more expensive side, this scores well for its ability to conduct quantitative analysis with the same granularity as qualitative analysis.

More and more players in the area of usability testing tools are entering the arena every day and each one has its own strengths and weaknesses. In my experience, I have found Usabilla very functional but ultimately, the best way to choose a tool is to find the one that best suits the research needs of the project design and development process of the organization.