Prototyping

Prototyping

by Dr. Bob Bailey

January, 1999

2 separate studies find no significant difference between high- and low-fidelity prototypes in usability testing.

Prototyping 1

One question that continues to be asked is how high does the fidelity level of a prototype need to be in order for the prototype to be useful in usability testing?

A study by Catani and Biers investigated the effect of prototype fidelity on the information obtained from a performance test. They had three levels of prototypes:

Paper – low fidelity
Screen shots – medium fidelity
Interactive Visual Basic – high fidelity

Thirty university students performed four typical library search tasks using one of the three prototypes. A total of 99 usability problems were uncovered. The mean number of total problems encountered was:

Low fidelity – 24.8
Medium fidelity – 29.4
High fidelity – 28.0

There were no significant differences in the number and severity of problems identified, and there was a high degree of commonality in the specific problems uncovered by users using the three prototypes.

These results seem to support the contention that one can extract as much information from a usability test using low-fidelity (paper) prototypes as from one using higher fidelity prototypes.

Prototyping 2

Another study reported by Uceta, Dixon and Resnick had 10 participants use either a paper-based prototype or a computer- based prototype. Each participant used the prototypes to order various food and beverage items. Both interfaces were exactly the same except for the medium of presentation, and both prototypes contained the same number of usability problems.

Both approaches enabled users to find most of the usability problems. The paper prototype group and the computer-based prototype group were statistically comparable. However, testing with paper prototypes took about 30% longer than with computer-based prototypes.

Once again, indepently from the previous study, no reliable differences were found between the prototyping methods for finding usability problems.

References

Catani, M.B. and Biers, D.W., Usability Evaluation and Prototype Fidelity: Users and Usability Professionals, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1331-1336, (1998).

Uceta, F.A. Dixon, M.A. and Resnick, M.L., Adding Interactivity to Paper Prototypes, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 506-511, (1998).