Usability Testing & Usability Software

Usability Testing & Usability Software

by Dr. Bob Bailey

December, 1998

Usability Testing

What is a good combination of usability testing methods?

Usability testing early on in the development life cycle attempts to find severe problems that can be fixed as soon as possible. To be most successful, it is important to apply the most effective evaluation techniques at each stage. No single usability test seems to be universally applicable. There is little published advice on when to use which tests, and which techniques work best together. Beer, Anodenko and Sears suggested the effective use of two usability testing methods: cognitive walkthroughs and think-aloud evaluations.

Cognitive walkthroughs require evaluators to step through the actions users would perform to accomplish tasks while looking for usability problems. The technique originally focused on learnability issues and was considered difficult to apply. The technique has been modified over the years to be more usable by individuals with limited formal training in user interface evaluation.

Think-aloud evaluations involve having users attempt to perform specific tasks while thinking out loud. The primary goal of these evaluations is to discover what usability problems exist. Preparing for think-aloud evaluations involves recruiting and scheduling representative users, and developing a set of tasks for the users to perform.

This article suggests the combining of the two techniques. Designers should do cognitive walkthroughs followed by think-aloud evaluations. These work well together because task descriptions developed for the cognitive walkthroughs can be converted directly into the task descriptions needed for think-aloud evaluations.

A study was conducted to evaluate the effectiveness of using these two techniques. They found that using cognitive walkthroughs early-on allowed the most serious problems to be identified and corrected. This was done without the need to use representative users. Later, think-aloud evaluations were used to find the remaining problems.

Usability Software

How can we substantially improve the user interface design process?

One of the most exciting advances in the usability field is the creation of software that will simulate the behavior of typical users. Eventually, this will allow designers to conduct initial, and very effective usability tests, without requiring real users to perform the tasks. One of the most promising new methods is a system named Executive Process Interactive Control (EPIC).

EPIC
(a) has a programmable production-rule cognitive processor,
(b) incorporates constraints from human performance research, and
(c) generates a specific sequence of perceptual, cognitive and motor activities required to perform each specific instance of the task.

To use the software, designers must specify a cognitive strategy and perceptual-motor processing parameters.

To ensure that the system behaves in a manner similar to actual users, many studies are being conducted to determine exactly how users respond in specific situations. In this study, Hornof and Kieras attempted to determine how users search for a known item in an unfamiliar menu. They made adjustments to their software until the new system responded to a menu selection task in exactly the same way as typical users.

This and other studies should eventually lead to a general purpose computer-based tool for evaluating the efficiency of user interfaces. User interface designers would provide a definition of a screen layout and a task, and the system would predict the time required for users to execute the task.

The development of formal evaluation methods, such as EPIC, has tremendous potential for significantly improving the quality of user interfaces.

References

Beer, T., Anodenko, T. and Sears, A., A Pair of Techniques for Effective Interface Evaluation: Cognitive Walkthroughs and Think-Aloud Evaluations, Proceedings of the Human Factors and Ergonomics Society 41st Annual Meeting, 380-384, (1997).

Hornof, A.J. and Kieras, D.E., Cognitive Modeling Reveals Menu Search is Both Random and Systematic, Proceedings of CHI ‘97, 107-114, (1997).