Remote usability testing: preparing for the worst-case scenario

One project I’ve been working on lately has involved a remote usability testing session that accommodates all the challenging resources: multiple channels (live screen tracking + audio recording + a test guideline + few data input paper forms), multiple Lo-Fi prototypes for different user roles, fixed time for each test with limited interval in between.

 

It’s likely a worst-case scenario from my experience, although some tasks have already been “reserved” back toward the user interview session. After a few iterations of operation and modification, ultimately it was carried out pretty well and I’ve had sufficient data for the outcome documentation.

Here are a couple of suggestions that summarize my experience.

Strategy

The focus of the test at early stage is structure and navigation, but not detailed interaction and visual design. Therefore, only key screens need to be covered, plus avoid testing participants on performance-driven interaction tasks.

Prototype

Break down the entire application design into user-role specified prototypes for distinct group of participants. Some important and/or common tasks can be reused across all prototypes in order to still get a bigger data coverage in case it’s cut off without adequate time.

Test guideline

It’s almost a must have for participant to hold a printout test guideline in the remote usability testing as they’re expected to efficiently understand the task with sufficient information. However, we don’t want them to take advantage of the context to counteract the intuitive reaction to the given tasks. Thus, the guideline should not be passed too early but 30 minutes prior to the test session.

Test task

Think aloud protocol becomes very helpful for moderator to understand participant’s intention from mouse tracks and actions without seeing their facial expressions. Since the tooltip feature is disabled within Lo-Fi prototype, it’s better to replace all iconic buttons with text-only ones.