UXPA Boston Conference 2020

Usability testing is by far the most widely used usability method. It’s important that we learn from experienced practitioners about this method so we can continue to improve our skills in usability testing and avoid common errors.

Through presentations and interactive quizzes based on video clips, this talk reports how 15 experienced usability professionals and two graduate students moderated usability tests. The purpose of the study was to investigate the approaches to moderation used by experienced professionals. Based on this study, the talk presents some of the characteristics that distinguish good and bad moderation.

Each moderator independently moderated three think-aloud usability test sessions of Ryanair.com, the website of a low-fare European airline. All moderators used the same six usability test tasks. The test sessions were video recorded so that both the participant and moderator were visible.

Key observations were identified by asking other study participants to review a random video from each moderator. Each video was reviewed by five to seven study participants. With this approach, the data, not a single person, author, organizer, or moderator, determines what the key observations are.

This study, which is the tenth in a series of Comparative Usability Evaluation (CUE) studies, documents a wide difference in moderation approaches. The talk presents important issues in usability test moderation, including time management, building trust and rapport, pilot sessions, design discussions, and how you can keep sharp.