With remote usability testing, one can conduct user research in his natural environment with participants through screen-sharing software or online remote usability services. Generally, the test is done with about 3-5 tasks on the app or the website and it takes about 15-30 minutes.
Remote usability tests can be “moderate” in the same way you would for an individual lab test or “un-moderate,” where participants complete tasks independently.
When to Consider Remote Testing
There are certain conditions when it is better to consider remote usability testing. Some of those conditions include:
- Scheduling issues can prevent individual testing
- The target participants are geographically dispersed making it difficult for them or users to travel
- Requires participants to use a particular device/app due to software or security requirements.
- Participants have accessibility issues that require them to use their own software or equipment.
- With its flexibility, you can potentially run multiple tests at once.
Although this method is flexible, like any test, you should consider the benefits and costs of this method.
Benefits
- It eliminates the need for a laboratory environment and the impact of the laboratory environment on participants.
- Accommodates diverse groups of participants.
- Generally less expensive than traditional in-person lab testing.
- In non-moderate testing, it allows you to extend your test day, possibly allowing you to reach a larger pool of participants.
- This is an opportunity to administer the test to a larger group of people, which you may be able to accommodate in a laboratory environment.
Challenges
- Security can be compromised when sensitive, privileged, or intellectual property is tested.
- Restrictions on user gestures or any visuals may impede certain indications of their reactions to the material being tested.
- Technical difficulties are likely to occur if the user:
- Not comfortable with the technology involved,
- Have conflicting software or devices on their machines,
- Unable to share your screen over the Internet,
- Have unreliable or slow connection speeds.
- If your trial requires a special device or software download or plug-in, participants may be reluctant or unable to download.
Selecting Your Participants
First, you need to know whether you will be provided with respondents for testing or choose a group of users yourself.
It would be great to have direct access to the users. If it’s impossible, then you should determine the target audience by some parameters (to be done optionally depending on goals and preferences):
- Age
- Gender
- Household Income
- Education
- Type of employment (full-time/part-time)
- Work area
- Company size
- Duty and Level (middle/head etc.)
- Location(address space)
- Web expertise (advanced/beginner PC users).
When performing remote testing, it is advisable to specify the Operating System (iOS/Windows/Android) and Device Type (Desktop/Mobile/Tablet) if necessary
Developing a Remote Usability Test
The process of developing remote testing is very similar to developing the test and materials for the testing you are doing in person. Often you can take advantage of a personal test that you will have run for use in a remote setting.
It is important to:
- Remember that the test should be about 15-30 minutes long consisting of about 3-5 tasks.
- Develop straight functions with clearly defined end states.
- If you are using a screener, make sure to include the minimum system requirements for the site or device you are testing, but also for the screen sharing service you offer to use.
- Make sure you have the correct contact information for your participants for reminders and follow-up if necessary.
- Prepare introductorily and test material so participants know what is expected of them as well as what they can expect from you.
- Prepare the test consent form.
- Prepare compensation and receipts for compensation if you choose to pay your participants.
Remote testing differs mainly in technology. You must know that:
- Anything you’re testing is accessible outside the firewall of your business, agency, or dev environment.
- You will need to determine if a firewall issue is affecting your participants’ ability to access the site or device you are testing.
- Participants can download/access screen-sharing software, or online remote utility vendor services.
Moderated Remote Usability Testing
During the moderated remote usability test, participants are observed interacting while completing tasks for the test. Moderate testing is best for complex tasks that do not have a structured sequence of steps or where testing would benefit from more interaction and questioning.
To perform a simple remote usability test, in addition to the general guidelines outlined above, you’ll need to:
- Define the test and decide what tasks you are going to ask participants to perform, the sequence of tasks, and any follow-up questions.
- schedule test session
- Prepare enough time in the test session to allow technical issues to be resolved.
During moderated remote usability tests you will be able to:
- Assign interview-based job performance
- ask to follow up questions
- Assure the completion of the test
- Test and analyze as planned
Depending on the technology or methodology used, you may be able to analyze metrics such as task completion rate, time on task, time on page, clickstream path, and satisfaction rating or opinion ranking.
Unmoderated Remote Usability Testing
During non-powered remote testing, participants complete the test independently without interaction with the test moderator. Non-supervised testing is most effective when you have very specific questions about how people use the UI for relatively simple tasks.
To perform an unattended remote usability test, in addition to the general guidelines outlined above, you’ll need to:
- Define the test and decide what tasks you are going to ask participants to perform, including the sequence of tasks.
- Decide who will participate. Many non-powered testing tools allow you to either intercept users on your Web site or recruit them from the tool developers’ own list of test participants.
- Before testing, determine whether your screen sharing service either has recording capability or will allow the use of your recording equipment or software.
- Keep in mind that if your software can only accept a single video input, then picture-in-picture (PIP) or webcam may not be captured.
- IWhen choosing from random participants, make sure they can represent your target audience.
- Include more qualitative research questions and analyze self-reported feedback.
- Focus on time-on-task and open-ended feedback to filter out participants who participate for the sole purpose of earning incentives
- If your participants have a vision challenge, consider asking if they are familiar with the screen share software you are using. If not, consider asking if a coworker or family member will be available during the test; to make sure they can share their screen
During an unsupervised remote usability test, you will need to:
- Start the test and send participants an email invitation to complete it
- You can analyze your results after the session is over. Many non-powered testing tools automatically generate an analysis for you.
Technology-based, non-powered remote usability testing allows you to analyze metrics such as task completion rate, time on task, time on page, clickstream path, satisfaction rating or opinion ranking, and web analytics data (e.g. browser, operating, system, and screen resolution).
The Right Type of Test will Bring You the Expected Results
Traditional usability testing is conducted in the laboratory, but remote usability testing allows you to test more participants for much less cost. When done correctly, it can be a secret weapon for teams that try and implement customer feedback.
Each user research method has different advantages and disadvantages. Understanding each approach’s unique strengths and limitations will ensure that you are getting the results you need to create better products and experiences for your customers.
