What Is Usability Testing?

This is part of a white paper I wrote in 2003:

Usability Testing has as its goal the analysis of the user experience by direct evaluation of actual users. A usability test ensures the collection of systematic, recorded, quantifiable data and observation of user behaviors. At it’s simplest level, usability testing will determine whether a system works well from the user’s perspective, or whether it requires rework.

Test Objectives:
Usability testing covers all user interfaces required to carry out the functions of the web site. Each project or system which is meant to interact with an actual user, whether internal employee carrying out a process, third party professional accessing specialized system areas, or individual users interacting with the web site is subjected to usability testing.

Usability is characterized by the following attributes:

Easy to learn. The user should be able to learn basic functions of the system quickly and the more advanced functions gradually. Example: the provision of menus, wizards, help files and accessible tutorials can make a system easier to learn by a less experienced user.

Easy to remember. A user should not have to relearn the system after being away from it for a while.

Efficient to use. Once a user has learned the system, a high level of productivity should be possible. Example: toolbar buttons and keyboard shortcuts for common activities are available and can be customized by the advanced user.

Few errors and easy recovery (Forgiveability). The system should help the user avoid errors and make recovery from errors quickly and easily. Example: confirmation windows on high impact commands, spell checkers, etc.
Subjectively pleasing. The system should be satisfying to use so that users are comfortable with it. These are primarily design considerations.

Test Design Strategy:
The Usability Lifecycle covers each phase of the creation and implementation of the web site (Architecture, Development, Integration, Deployment and Operations). It has specific activities to be carried out and work products which are created by those activities.

Characteristics of Usability tests:
Usability tests conducted after development and prior to release are primarily Validation tests. The activities listed below are designed to obtain user feedback through Usability testing, thereby producing elements for the Usability Review. They meet the following criteria:

Each participant completes an interview/questionnaire.

Each participant will receive a short, verbal orientation to the project site, an explanation of the test purpose, and a statement on how they will be observed and taped. They must also sign a video release.

Each participant performs their assigned task or tasks.

A final interview is held with the participant.

The data is analyzed (both qualitatively – comments of observers- and quantitatively – metrics involving time to completion of tasks, number of errors, etc.), problems diagnosed, and recommendations made.

Usability Review is finalized.

Test Environment Planning:
Usability testing requires specific lab configurations and testing equipment. In most cases, this is a fully compatible laboratory environment consisting of: An enclosed room, video cameras and recorders for both video and sound, one way mirrors for observers to monitor tests (or closed circuit video for monitoring), user workstation(s) serving as a model of the user’s actual work area.