Abstract
Inspection is a fundamental means of achieving software usability. Past research showed that the current usability inspection techniques were rather ineffective. We developed perspective-based usability inspection, which divides the large variety of usability issues along different perspectives and focuses each inspection session on one perspective. We conducted a controlled experiment to study its effectiveness, using a post-test only control group experimental design, with 24 professionals as subjects. The control group used heuristic evaluation, which is the most popular technique for usability inspection. The experimental design and the results are presented, which show that inspectors applying perspective-based inspection not only found more usability problems related to their assigned perspectives, but also found more overall problems. Perspective-based inspection was shown to be more effective for the aggregated results of multiple inspectors, finding about 30% more usability problems for 3 inspectors. A management implication of this study is that assigning inspectors more specific responsibilities leads to higher performance. Internal and external threats to validity are discussed to help better interpret the results and to guide future empirical studies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Basili, V. 1997. Evolving and packaging reading technologies. Journal of Systems and Software, 38: 3–12.
Basili, V., Green, S., Laitenberger, O., Shull, F., Sorumgard, S., and Zelkowitz, M. 1996. The empirical investigation of perspective-based reading. Empirical Software Engineering, 1.2:133–164.
Campbell, D. T., and Stanley. J. C. 1966. Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin Company.
Desurvire, H. W. 1994. Faster, cheaper!! Are usability inspection methods as effective as empirical testing? In J. Nielsen and R. L. Mack (Eds.), Usability Inspection Methods, chapter 7, pages 173–202. John Wiley & Sons, Inc.
Desurvire, H. W., Kondziela, J. M., and Atwood, M. E. 1992. What is gained and lost when using evaluation methods other than empirical testing. In A. Monk, D. Diaper, and M. D. Harrison (Eds.), People and Computers VII, pages 89–102. Cambridge University Press.
Dumas, J. S., and Redish, J. C. 1993. A Practical Guide to Usability Testing. Norwood, NJ: Ablex Publishing Corporation.
Edington, E. S. 1987. Randomization Tests. New York: Marcel Dekker Inc.
Jeffries, R., Miller, J. R., Wharton, C., and Uyeda, K. M. 1991. User interface evaluation in the real world: A comparison of four methods. In ACM CHI'91 Conference Proceedings, pages 261–266. ACM.
Kurosu, M., Sugizaki, M., and Matsura, S. 1998. Structured heuristic evaluation. In Proceedings of the Usability Professionals' Association Conference, pages 3–5.
Nielsen, J. 1990. Paper versus computer implementations as mockup scenarios for heuristic evaluation. In D. Diaper et al. (Ed.), Human-Computer Interaction-INTERACT'90, pages 315–320. IFIP.
Nielsen, J. 1993. Usability Engineering. San Diego: Academic Press, Inc.
Nielsen, J. 1994. Heuristic evaluation. In J. Nielsen and R. Mack (Eds.), Usability Inspection Methods, chapter 2, pages 25–62. John Wiley.
Nielsen, J., and Mack, R. (Eds.). 1994. Usability Inspection Methods. John Wiley & Sons, Inc.
Norman, D. A. 1988. The Design of Everyday Things, 1st doubleday/currency edition. New York: Basic Books.
Porter, A. A., Votta, L. G. Jr., and Basili, V. R. 1995. Comparing detection methods for software requirements inspections: A replicated experiment. IEEE Transactions on Software Engineering, 21.6:563–575.
Rieman, J., Divies, S. Hair, D. C., and Esemplare, M. 1991. An automated cognitive walkthrough. In ACM CHI'91 Conference Proceedings, pages 427–428.
Seaman, C. B., and Basili, V. R. 1997. An empirical study of communication in code inspections. In Proceedings of the 19th International Conference on Software Engineering, pages 96–106.
Sears, A. 1997. Heuristic walkthroughs: Finding the problems without the noise. International Journal of Human Computer Studies, 9.3:213–234.
Shneiderman, B. 1998. Designing the User Interface: Strategies for Effective Human-Computer Interaction, 3rd edition. Addison-Wesley.
Tetzlaff, L., and Schwartz, D. R. 1991. The use of guidelines in interface design. In ACM CHI'91 Conference Proceedings, pages 329–333. ACM.
Thovtrup, H. and Nielsen, J. 1991. Assessing the usability of a user interface standard. In ACM CHI'91 Conference Proceedings, pages 335–341. ACM.
Wharton, C., Rieman, J., Lewis, C., and Polson, P. 1994. The cognitive walkthrough method: A practitioner's guide. In J. Nielsen and R. L. Mack (Eds.), Usability Inspection Methods, chapter 5, pages 105–140. John Wiley & Sons, Inc.
Wright, P. C., and Monk, A. F. 1991. A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies, 35:891–912.
Zhang, Z., Basili, V., and Shneiderman, B. 1998. Perspective-based usability inspection. In Proceedings of the Usability Professionals' Association Conference, pages 281–282.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Zhang, Z., Basili, V. & Shneiderman, B. Perspective-based Usability Inspection: An Empirical Validation of Efficacy. Empirical Software Engineering 4, 43–69 (1999). https://doi.org/10.1023/A:1009803214692
Issue Date:
DOI: https://doi.org/10.1023/A:1009803214692