Toshio Nakamura
Takeo Igarashi
ABSTRACT
A history-of-user-operations function helps make applications easier to use. For example, users may have access to an operation history list in an application to undo or redo a past operation. To provide an overview of a long operation history and help users find target interactions or application states quickly, visual representations of operation history have been proposed. However, most previous systems are tightly integrated with target applications and difficult to apply to new applications. We propose an application-independent method that can visualize the operation history of arbitrary GUI applications by monitoring the input and output GUI events from outside of the target application. We implemented a prototype system that visualizes operation sequences of generic Java Awt/Swing applications using an annotated comic strip metaphor. We tested the system with various applications and present results from a user study.
Supplemental Material
- Agrawala, M., Phan, D., Heiser, J., Haymaker, J., Klingner, J., Hanrahan, P., and Tversky, B. Designing Effective Step-By-Step Assembly Instructions. In Proceedings of SIGGRAPH, pp. 828--837, 2003. Google Scholar
Digital Library
- Agrawala, M., and Stolte, C. Rendering Effective Route Maps: Improving Usability Through Generalization. In Proceedings of SIGGRAPH, pp. 241--249, 2001. Google ScholarDigital Library
- Assa, J., Caspi, Y., and Cohen-Or, D. Action Synopsis: Pose Selection and Illustration. In Proceedings of SIGGRAPH, pp. 667--676, 2005. Google ScholarDigital Library
- Baudisch, P., Cutrell, E., and Robertson, G. High-Density Cursor: A Visualization Technique that Helps Users Keep Track of Fast-Moving Mouse Cursors. In Proceedings of INTERACT, pp. 236--243, 2003.Google Scholar
- Baudisch, P., Tan, D., Collomb, M., Robbins, D., Hinckley, K., Agrawala, M., Zhao, S., and Ramos, G. Phosphor: Explaining Transitions in the User Interface Using Afterglow Effects. In Proceedings of UIST, pp.169--178, 2006. Google ScholarDigital Library
- Bergman, L., Castelli, V., Lau, T., and Oblinger, D. DocWizards: a system for authoring follow-me documentation wizards. In Proceedings of UIST, pp. 191--200, 2005. Google ScholarDigital Library
- Berlage, T. A Selective Undo Mechanism for Graphical User Interfaces Based On Command Objects. In Proceedings of CHI, pp. 269--294, 1994. Google ScholarDigital Library
- Bezerianos, A., Dragicevic, P., and Balakrishnan, R. Mnemonic Rendering: An Image-Based Approach for Exposing Hidden Changes in Dynamic Displays. In Proceedings of UIST, pp. 159--168, 2006. Google ScholarDigital Library
- Freeman, W.T., and Zhang, H. Shape-Time Photography. In Proceedings of CVPR, pp. 151--157, 2003.Google Scholar
- Goldman, D.B., Curless, B., Salesin, D., and Seitz, S.M. Schematic Storyboarding for Video Visualization and Editing. In Proceedings of SIGGRAPH, pp. 862--871, 2006. Google ScholarDigital Library
- Guimbretie're, F., Dixon, M., and Hinckley, K. Experi-Scope: an analysis tool for interaction data. In Proceedings of CHI, pp.1333--1342, 2007. Google ScholarDigital Library
- Igarashi, T., Matsuoka, S., and Tanaka, T. Teddy: A Sketching Interface for 3D Freeform Design. In Proceedings of SIGGRAPH, pp. 409--416, 1999. Google ScholarDigital Library
- Irani, M., and Anandan, P. Video Indexing Based on Mosaic Representations. IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 86, No. 5, pp. 905--921, 1998.Google Scholar
- Kaptelinin, V., Mantyla, T., and Astrom, J. Transient Visual Cues for Scrolling: An Empirical Study. In CHI '02 Extended Abstracts, pp. 620--621, 2002. Google ScholarDigital Library
- Kawagishi, Y., Hatsuyama, K., and Kondo, K. Cartoon Blur: Non-Photorealistic Motion Blur. In Proceedings of CGI, pp. 276--281, 2003.Google Scholar
- Kim, B., and Essa, I. Video-based Nonphotorealistic and Expressive Illustration of Motion. In Proceedings of the CGI, pp. 32--35, 2005. Google ScholarDigital Library
- Kurlander, D. and Feiner, S. A history-based macro by example system. In Proceedings of UIST, pages 99--106, 1992. Google ScholarDigital Library
- Lieberman, H. Mondrian: A Teachable Graphical Editor. In Watch What I Do: Programming by Demonstration, pp. 341--358, 1993. Google ScholarDigital Library
- Little, G., Lau, T. A., Cypher, A., Lin, J., Haber, E. M., and Kandogan, E., Koala: capture, share, automate, personalize business processes on the web, In Proceedings of CHI, pp. 943--946, 2007. Google ScholarDigital Library
- Massey, M., and Bender, W. Salient Stills: Process and Practice. IBM Systems Journal, Vol. 35, No.3--4, pp. 557--573, 1996. Google ScholarDigital Library
- Masuch, M., Schlechtweg, S., and Schulz, R. Speedlines: Depicting Motion in Motionless Pictures. In SIGGRAPH '99 Conference Abstracts and Applications, pp. 277, 1999. Google ScholarDigital Library
- Meng, C., Yasue, M., Imamiya, A., and Mao, X. Visualizing Histories for Selective Undo and Redo. In Proceedings of APCHI, pp. 459, 1998. Google ScholarDigital Library
- Modugno, F. and Myers, B. A. Pursuit: graphically representing programs in a demonstrational visual shell. In Conference Companion of CHI, pp.455--456, 1994. Google ScholarDigital Library
- Myers, B.A., McDaniel, R.G., Miller, R.C., Ferrency, A.S., Faulring, A., Kyle, B.D., Mickish, A., Klimovitski, A., and Doane, P. The Amulet Environment: New Models for Effective User Interface Software Development. IEEE Transaction on Software Engineering, Vol. 23, No. 6, pp. 347--365, 1997. Google ScholarDigital Library
- Mynatt, E.D., Igarashi, T., Edwards, W.K., and La-Marca, A. Flatland: new dimensions in office whiteboards. In Proceedings of CHI, pp. 346--353, 1999. Google ScholarDigital Library
- Rekimoto, J. Time-Machine Computing: A Time-Centric Approach for the Information Environment. In Proceedings of UIST, pp. 45--54, 1999. Google ScholarDigital Library
- Su, S. Visualizing, Editing, and Inferring Structure in 2D Graphics, UIST 2007 Doctoral Symposium, 2007.Google Scholar
- Taniguchi, Y., Akutsu, A., and Tonomura, Y. PanoramaExcerpts: Extracting and Packing Panoramas for Video Browsing. In Proceedings of MULTIMEDIA, pp. 427--436, 1997. Google ScholarDigital Library
- Teodosio, L., and Bender, W. Salient Video Stills: Content and Context Preserved. In Proceedings of MULTIMEDIA, pp. 39--46, 1993. Google ScholarDigital Library
Index Terms
- An application-independent system for visualizing user operation history
Recommendations
User interface history
CHI EA '08: CHI '08 Extended Abstracts on Human Factors in Computing SystemsUser Interfaces have been around as long as computers have existed, even well before the field of Human-Computer Interaction was established. Over the years, some papers on the history of Human-Computer Interaction and User Interfaces have appeared, ...
Taking stock of user interface history
NordiCHI '08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridgesThe user interface is coming of age and papers adressing UI history have appeared in fair amounts in the last 25 years. Most of them address particular aspects such as an innovative interface paradigm or the contribution of a visionary or a research ...
Comments