Philip Cohen
David McGee
ABSTRACT
This paper compares the efficiency of using a standard direct-manipulation graphical user interface (GUI) with that of using the QuickSet pen/voice multimodal interface for supporting a military task. In this task, a user places military units and control measures (e.g., various types of lines, obstacles, objectives) on a map. Four military personnel designed and entered their own simulation scenarios via both interfaces. Analyses revealed that the multimodal interface led to an average 3.5-fold speed improvement in the average entity creation time, including all error handling. The mean time to repair errors also was 4.3 times faster when interacting multimodally. Finally, all subjects reported a strong preference for multimodal interaction. These results indicate a substantial efficiency advantage for multimodal over GUI-based interaction during map-based tasks.
- Chapanis, A., Ochsman, R. B., Parrish, R. N., Weeks, G. D., Studies in interactive communication: I. The effects of four communication modes on the behavior of teams during cooperative problem solving. Human Factors, 1972. 14: pp. 487--509.Google Scholar
Cross Ref
- Cohen, P. R., Dalrymple, M., Moran, D. B., Pereira, F., Sullivan, J., Gargan, R., Schlossberg, J., and Tyler, S., Synergistic use of natural language and direct manipulation, in Proc. of the Human-Factors in Computing Systems Con-ference (CHI'89). 1989, ACM Press: New York, pp. 227--234. Google Scholar
- Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Clow, J., and Smith, I., The efficiency of multimodal interaction: A case study, in the Proceedings of the 5th International Conference on Spoken Language Processing, Sydney, Australia, 1998, 2: pp. 249--252.Google Scholar
- Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., Clow, J., QuickSet: Multimodal interaction for distributed applications, in Proc. of the Fifth ACM International Multmedia Conference, E. Glinert, Editor. 1997, ACM Press: New York. pp. 31--40. Google ScholarDigital Library
- Cohen, P.R., McGee, D., Oviatt, S., Wu, L., Clow, J., King, R., Julier, S., Rosenblum, L., Multimodal Interaction for 2D and 3D Environments. IEEE Computer Graphics and Applications, 1999. 19(4): pp. 10--13. Google ScholarDigital Library
- Cohen, P.R. and Oviatt, S.L., The Role of Voice Input for Human-Machine Communication. Proc. of the National Academy of Sciences, 1995. 92: pp. 9921--9927.Google ScholarCross Ref
- Courtemanche, A.J., Ceranowicz, A., ModSAF Development Status., in the Proc. of the Fifth Con-ference on Computer Generated Forces and Behavioral Representation, Orlando, 1995, Univ. of Central Florida, pp. 3--13.Google Scholar
- Johnston, M., Cohen, P. R., McGee, D., Oviatt, S. L., Pittman, J. A., Smith., I. Unification-based multimodal integration., in the Proc. of the 35th Annual Meeting of the Association for Computational Linguistics (ACL) and 8th Conference of the European Chapter of the ACL, 1997, pp. 281--288. Google ScholarDigital Library
- Karat, C., Halverson, C., Horn, D., and Karat, J., Patterns of entry and correction in large vocabulary continuous speech recognition systems, in the Proc. of Human Factors in Com-puting Systems, New York, 1999, ACM Press, pp. 568--575. Google ScholarDigital Library
- Martin, G.L., The utility of speech input in user-computer interfaces. International Journal of Man-machine Studies, 1989. 30(4): pp. 355--375. aR@<11>McGee, D., Cohen, P.R., and Oviatt, S.L., Confirmation in Multimodal Systems, in Proc. of the 17th International Conference on Computational Linguistics (COLING '98) and 36th Annual Meeting of the Association for Computational Linguistics (ACL'98). 1998: Montreal, Canada. pp. 823--829. Google ScholarDigital Library
- Moore, R., Dowding, J., Bratt, H., Gawron, J., Gorfu, Y., Cheyer, A., CommandTalk: A Spoken-Language Interface for Battlefield Simulations, Proc. of the 5th Conference on Applied Natural Language Processing, Association for Computational Linguistics, 1997: Washington, DC. pp. 1--7. Google ScholarDigital Library
- Oviatt, S. L., Pen/Voice: Complementary multimodal communication, Proc. of Speech Tech'92, New York, 238--241Google Scholar
- Oviatt, S.L., Multimodal interactive maps: Designing for human performance. Human Computer Interaction, 1997. 12: pp. 93--129. Google ScholarDigital Library
- Oviatt, S.L., Mutual disambiguation of recognition errors in a multimodal architecture, in the Proc. of the Conference on Human Factors in Computing System, New York, 1999, ACM Press, pp. 576--583. Google ScholarDigital Library
- Oviatt, S.L., Cohen, P. R., Wu, L., Vergo, J., Duncan, L., Suhm, B., Bers, J., Holzman, T., Winograd, T., Landay, J., Larson, J., Ferro, D., Designing the user interface for multimodal speech and gesture applications: State-of-the-art systems and research directions for 2000 and beyond. In submission.Google Scholar
- Pausch, R. and Leatherby, J. H., A study comparing mouse-only vs. mouse-plusvoice input for a graphical editor, Journal of the American Voice Input/Output Society, 9:2, July, 1991, pp 55--66Google Scholar
- Rudnicky, A.I., Mode Preference in a simple data-retrieval task, in ARPA Human Language Technology Workshop. March 1993: Princeton, New Jersey. Google ScholarDigital Library
- Suhm, B., Myers, B., and Waibel, A., Model-based and empirical evaluation of multimodal interactive error correction, in the Proc. of the Conf. on Human Factors in Computing Systems, New York, 1999, ACM Press, 584--591. Google ScholarDigital Library
- Shneiderman, B., Natural vs. precise concise languages for human operation of computers: Research issues and experimental approaches. Proceedings of the 18th Annual Meeting of the Association for Computational Linguistics, and Parasession on Topics in Interactive Discourse, Univ. of Pennsylvania, June, 1980, pp. 139--141. Google ScholarDigital Library
- Wickens, C., Sandry, D., and Vidulich, M., Compatibility and resource competition between modalities of input, central processing, and output. Human Factors, 1983. 25(2): pp. 227--248.Google ScholarCross Ref
- Wu, L., Oviatt, S., L. and Cohen, P. R., Statistical multimodal integration for intelligent HCI, in Neural Networks for Signal Processing, Y.H. Hu, Larsen, J., Wilson, E., and Douglas, S., Editors. 1999, IEEE Press: New York. pp. 487--496.Google Scholar
- The efficiency of multimodal interaction for a map-based task
Recommendations
The efficiency of multimodal interaction for a map-based task
CHI EA '00: CHI '00 Extended Abstracts on Human Factors in Computing SystemsThis paper compares the efficiency of using a standard direct-manipulation graphical user interface (GUI) with that of using the QuickSet pen/voice multimodal interface for supporting a military task. In this task, a user places military units and ...
Efficiency of multimodal metaphors in e-learning platform
SEPADS'10: Proceedings of the 9th WSEAS international conference on Software engineering, parallel and distributed systemsThe primary focus of this paper is to investigate the effect of including multimodal metaphors on the usability of e-learning interfaces. This investigation involved two different interface platforms performed by forty users. The first interface ...
Comments