Expert Reviews, Usability Testing, and User Research – What’s the Difference?

It’s true that more people than ever are aware of user experience these days, but I still find there are a lot of misunderstandings about the difference between these methods. So let’s explore each of these methods, the types of findings they produce, and when you would use one over the other.

Expert Review

What It Is

Expert reviews involve a user experience expert evaluating an existing user interface, such as software or a website, by applying his or her knowledge of human factors, usability guidelines, heuristics, design principles, information architecture best practices, and accessibility standards. The result is a prioritized list of problems and recommended solutions.

Results

Expert reviews typically find very tactical problems that can be immediately addressed to improve the user interface. It’s useful in finding violations of design and usability best practices, but since it doesn’t involve users, it can’t find the types of problems that only can be found by people using the system. For example, without involving users, it can’t determine whether the terminology will be understandable or whether the work flow matches the way they perform tasks.

Advantages

• Only requires one or two experts
• Quickest and shortest method
• Least expensive
• No users are involved, so you avoid the cost, effort, and time involved in recruiting, scheduling, and running sessions

Disadvantages

• The quality and reliability of the findings and recommendations depends on the quality, knowledge, and experience of the expert
• Doesn’t involve users, so it can’t find problems that are only found by the end users of the system

When to use it

Expert reviews are a good first step to clean up an interface. They’re not a substitute for testing an application with users, but they are good at finding and eliminating the basic problems.

Usability Testing

What it Is

Usability testing involves asking people to perform typical tasks using the application in question, and record a combination of objective and subjective measures such as completion rate, time to complete, errors, and how users feel about the experience. Testing the application with a number of participants in individual sessions reveals common problems with the interface.

Results

Like an expert review, usability testing provides a list of very specific problems and recommended solutions with regard to an existing interface. The primary advantage of usability testing is that instead of an expert evaluating the interface; it’s tested by actual users. While most of the findings are very tactical, testing can also reveal more strategic problems with the interface.

Advantages

• The interface is evaluated by actual users, which makes the findings more relevant to improving the experience of end users
• Finds problems that only the end users of the application can find

Disadvantages

• Requires the time, cost, and effort to recruit, schedule, and run sessions with participants
• Only finds problems based on the tasks you test, which might not cover the full spectrum of how people would actually use the application
• It doesn’t provide in-depth information about the users and the context in which they perform their tasks

When to use it

Usability testing can be used to evaluate an early design concept, a prototype, or a completed interface. Ideally, usability testing should be used throughout a design process to evaluate a design and make changes based on the problems found.

User Research

What it Is

There are many different methods, but typical user research involves interviewing participants and observing them perform typical tasks. The goal is to understand the users’ context of use, which includes the users’ characteristics, their tasks, tools and technology, and the environment in which they perform those tasks with the purpose of understanding the requirements of a yet-to-be-built application.

Results

User research provides higher-level, strategic information about users and their needs. It generates user requirements that are then used to design an effective user experience.

Advantages

• Provides the most realistic understanding of the users, their context of use, and their needs
• No designs or prototypes are needed as it is typically conducted at the beginning of a project
• Knowledge of the users and their tasks makes it easier to design an effective solution that fits their needs

Disadvantages

• Requires the time, cost, and effort to recruit, schedule, and run sessions with participants

When to use it

User research is best performed at the beginning of a project to understand the users and their tasks in order to use that information to design an effective solution that meets their needs.

As a summary, the following table shows the differences between these methods:

	Results	Advantages	Disadvantages	When to Use
Expert Review	Identifies specific, tactical usability and design problems and recommends solutions	Only requires one or two experts Quick Least expensive No time, cost, and effort to recruit participants	Quality and reliability depends on the experience of the expert Does not provide feedback from actual users	To find and eliminate basic design and usability issues.
Usability Testing	Identifies usability and design problems and recommends solutions	Involves actual users Finds problems that can only be found by the users	Requires the time, cost, and effort of recruiting participants Only finds problems related to the tasks you test Does not provide information about the users and their context of use	To evaluate design concepts, prototypes, and completed interfaces in an iterative design process
User Research	Provides strategic information about users, their context of use, and their requirements	Provides the most realistic understanding of the users, their context of use, and their needs This knowledge makes it easier to design an effective solution	Requires the time, cost, and effort to recruit, schedule, and run sessions with participants	At the beginning of a project to start with an understanding of the users and their needs

 

How to Best Combine These Methods

In the ideal design process, user research is conducted at the beginning of a project, to understand the users and their context of use. Based on that understanding, the interface is designed and usability testing is conducted throughout the design process; with early concepts, with early designs, and with prototypes in an iterative process of testing and design changes based on the findings.

To evaluate an existing interface, an expert review can be conducted first to find and correct the basic problems. Then it can be tested with users in usability testing to evaluate the types of issues that only users can find.

These three methods, used together at the right times, are powerful ways to ensure a good user experience.

Image courtesy of Alexander Henning Drachmann on Flickr