Heuristic Evaluations

Heuristic Evaluations

by Dr. Bob Bailey

June, 1999

Is there a useful set of usability heuristics currently available to practitioners? There is, but unfortunately the best set is not the one most widely used.

Heuristic evaluations are conducted by a small set of evaluators. The evaluators examine a user interface, and judge its compliance with a set of internalized usability principles. Usually, evaluators try to estimate the degree to which each usability problem would potentially impede user performance or acceptance. This is done to help set priorities for making revisions to the system.

Theoretically, the heuristics that are used are related to recognized usability criteria that, if improved, can make a positive difference in the product's usability. Unfortunately, "usability problems" are frequently identified that differ substantially from those obtained in performance testing (Catani and Biers, 1998). Part of the problem could be that evaluators continue to rely on faulty sets of heuristics.

Several recent papers referenced a set of usability heuristics as though these heuristics had some validity in the evaluation of user interfaces.

The most-used set of heuristics, shown below, can be traced back to a ten-year old paper by Molich and Nielsen (1990):

Use simple and natural dialogue
Speak the user's language
Minimize the user's memory load
Be consistent
Provide feedback
Provide clearly marked exits
Provide shortcuts
Provides good error messages
Prevent errors
Provide help and documentation

These heuristics, which are widely used, have never been validated. There is no evidence that by applying these heuristics in the design and development of user interfaces that it will improve the interface.

In fact, Nielsen (1994), after evaluating several sets of heuristics, concluded that a better set of heuristics may be:

Visibility of system status
Match between system and the real world
User control and freedom
Consistency and standards
Error prevention
Recognition rather than recall memory
Flexibility and efficiency of use
Aesthetic and minimalist design
Helping users recognize, diagnose and recover from errors

An even better, research-based, set of heuristics was proposed by Gerhardt-Powals (1996):

Automate unwanted workload
- Free cognitive resources for high-level tasks
- Eliminate mental calculations, estimations, comparisons, and unnecessary thinking
Reduce uncertainty
- Display data in a manner that is clear and obvious
Fuse data
- Reduce cognitive load by bringing together lower level data into a higher level summation
Present new information with meaningful aids to interpretation - Use a familiar framework, making it easier to absorb
- Use everyday terms, metaphors, etc.
Use names that are conceptually related to function
- Context-dependent
- Attempt to improve recall and recognition
Group data in consistently meaningful ways to decrease search time
Limit data-driven tasks
- Reduce the time spent assimilating raw data
- Make appropriate use of color and graphics
Include in the displays only that information needed by the user at a given time
- Allow users to remain focused on critical data
- Exclude extraneous information that is not relevant to current tasks
Provide multiple coding of data when appropriate
Practice judicious redundancy (to resolve the possible conflict between heuristics 6 and 8)

Unfortunately, the best potential set of heuristics is little known and little used. And even worse, a much weaker set is widely known and apparently much used. Is it any wonder that we do not have more of an impact on the quality of new user interfaces.

References

Catani, M.B. and Biers, D.W., Usability evaluation and prototype fidelity: users and usability professionals, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, (1998).

Gerhardt-Powals, J., Cognitive engineering principles for enhancing human-computer performance, International Journal of Human-Computer Interaction, 8(2), 189-211, (1996).

Jacobsen, N. E., Hertzum, M., and John, B. E., The evaluator effect in usability studies: problem detection and severity judgments, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1336-1340, (1998).

Molich, R. and Nielsen, J., Improving a human- computer dialogue, Communications of the ACM, 33(3), 338-348, (1990).

Nielsen, J., Enhancing the explanatory power of usability heuristics, CHI'94 Conference Proceedings, (1994).

Zhang, Z., Basili, V., and Shneiderman, B., An empirical study of perspective-based usability inspection, Proceedings of the Human Factors and Ergonomics society 42nd Annual Meeting, (1998).