Skip to main content
SpringerLink
Log in

Visual language parsing: If I had a hammer...

  • Conference paper
  • First Online:
Multimodal Human-Computer Communication (CMC 1995)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1374))

Included in the following conference series:

  • 278 Accesses

Abstract

Since the 1960s, grammatical formalisms and parsing methods developed originally for natural language strings have been extended to represent and process two-dimensional visual expressions such as mathematics notation and various kinds of diagrams. But despite all of the effort, there has been negligible impact on human-computer interfaces to support visual modes of communication. Why? As with all tech transfer issues, some of the reasons may be beyond a researcher's control. However, I believe that two of the contributing factors in the case of visual language (VL) parsing can and should be addressed by the research field. First, the field needs to consolidate and communicate its results. This is in fact not trivial for higher-dimensional visual language representation and parsing, and I will try to illustrate why. Second, researchers have to look harder for the right application domains. One of the obvious applications is the interpretation of visual language expressions constructed with GUIs. While grammatical representation and parsing may bring something to the table, the problem of interpretation may be solvable with simpler techniques. I will discuss some other application areas and my experience with them: design support, smart screen layout for electronic publishing, and visual focusing for attributed graphs.

The original version of this paper was written while the author was with Bellcore, Morristown, NJ, USA.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aho, A., and Johnson, S.C. (1974), Programming Utilities and Libraries — LR Parsing, Computing Surveys, June 1994.

    Google Scholar 

  • Aho, A., Sethi, R., and Ullman, J. (1986), Compilers: Principles, Techniques, and Tools, Addison-Wesley.

    Google Scholar 

  • Anderson, R.H. (1968), Syntax-Directed Recognition of Hand-Printed Two-Dimensional Mathematics. In Interactive Systems for Experimental Applied Mathematics, Klerer, M., and Reinfelds, J. (eds.), Academic Press.

    Google Scholar 

  • Backlund, B., Hagsand, O., and Pehrson, B. (1990), Generation of Visual Language-oriented Design Environments, Journal of Visual Languages and Computing 1, pp. 333–354.

    Article  Google Scholar 

  • Baird, H. (1990), Industrial Applications. In Syntactic and Structural Pattern Recognition: Theory and Applications, Bunke, H. and Sanfeliu, A. (eds.), World Scientific, pp. 369–380.

    Google Scholar 

  • Brandenburg, F. 1994. Designing Graph Drawings by Layout Graph Grammars. In Graph Drawing: DIMACS International Workshop, R. Tamassia and I.G. Tollis (eds.), Lecture Notes in Computer Science 894, Springer-Verlag, pp. 416–427.

    Google Scholar 

  • Brandenburg, F. (1989), On Polynomial Time Graph Grammars. In STACS 88: 5th Annual Symposium on Theoretical Aspects of Computer Science, Goos, G., and Hartmanis, J. (eds.), Lecture Notes in Computer Science 294, Springer-Verlag, pp. 227–236.

    Google Scholar 

  • Bunke, H. (1982), Attributed Programmed Graph Grammars and their Application to Schematic Diagram Interpretation, IEEE Transactions on Pattern Analysis and Machine Intelligence 4, PP. 574–582.

    Article  Google Scholar 

  • Chang, S.-K. (1988), The Design of a Visual Language Compiler. In Proceedings of IEEE Workshop on Visual Languages, Pittsburgh, Pennsylvania, USA, pp. 84–91.

    Google Scholar 

  • Chok, S.S., and Marriott, K. (1995), Automatic Construction of User Interfaces from Constraint Multiset Grammars. In Proceedings of IEEE Workshop on Visual Languages, Darmstadt, Germany, pp. 90–98.

    Google Scholar 

  • Costagliola, G., Tomita, M., and Chang, S.K. (1991), A Generalized Parser for 2-D Languages. In Proceedings of IEEE Workshop on Visual Languages, Kobe, Japan, pp. 98–104.

    Google Scholar 

  • Cox, P.T., Giles, F.R., and Pietrzykowsky, T. (1989), Prograph: a step towards liberating programming from textual conditioning. In Proceedings of IEEE Workshop on Visual Languages, Rome, Italy.

    Google Scholar 

  • Crimi, A., Guercio, A., Nota, G., Pacini, G., Tortora, G., and Tucci, M. (1991), Relation Grammars and their Application to Multi-dimensional Languages, Journal of Visual Languages and Computing 2, pp. 333–346.

    Article  Google Scholar 

  • Cuny, J. (1995), Graph Grammars and Their Application to Computer Science: 5th International Workshop, Williamsburg, Va, Usa, Springer-Verlag.

    Google Scholar 

  • Fu, K.-S. (1974), Syntactic Methods in Pattern Recognition, Academic Press.

    Google Scholar 

  • Golin, E., and Reiss, S. (1989), The Specification of Visual Language Syntax. In Proceedings of IEEE Workshop on Visual Languages, Rome, Italy, pp. 105–10.

    Google Scholar 

  • Golin, E. (1991), Parsing Visual Languages with Picture Layout Grammars, Journal of Visual Languages and Computing 2, pp. 371–393.

    Article  Google Scholar 

  • Gross, M. (1994), Recognizing and Interpreting Diagrams in Design. In Proceedings of Advanced Visual Interfaces (AVI), Catarci, T. Costabile, M., Levialdi, S., and Santucci, G. (eds).

    Google Scholar 

  • Helm, R., and Marriott, K. (1991), A Declarative Specification and Semantics for Visual Languages, Journal of Visual Languages and Computing 2, pp. 311–331.

    Article  Google Scholar 

  • Jagadeesh, J. and Wang, Y. (1993), Labview, Computer, February 1993.

    Google Scholar 

  • Joshi, A. (1985), Tree Adjoining Grammars: How Much Context-Sensitivity is Required to Provide Reasonable Structural Descriptions? In Perspectives, Dowty, D., Karttunen, L., and Zwicky, A. (eds.), Cambridge University Press, pp. 206–250.

    Google Scholar 

  • Kahn, K. and Saraswat, V. (1990), Complete Visualizations of Concurrent Programs and their Executions. In Proceedings of IEEE Workshop on Visual Languages, Skokie, Illinois, USA, pp. 7–14.

    Google Scholar 

  • Kramer, A. (1994), Translucent Patches-Dissolving Windows. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), pp. 121–130.

    Google Scholar 

  • Landay, J. and Myers, B. (1995), Interactive Sketching for the Early Stages of User Interface Design. In Proceedings of ACM Conference on Human Factors in Computing (CHI), pp. 43–50.

    Google Scholar 

  • Marriott, K. and Meyer, B. (1997), The CCMG Visual Language Hierarchy. In Theory of Visual Languages, Marriott, K. and Meyer, B. (eds.), Lecture Notes in Computer Science, Springer Verlag (in press).

    Google Scholar 

  • Marriott, K., Meyer, B., and Wittenburg, K. (1997), Survey: Visual Language Specification and Recognition. In Theory of Visual Languages, Marriott, K. and Meyer, B. (eds.), Lecture Notes in Computer Science, Springer Verlag (in press).

    Google Scholar 

  • Meyer, B. (1992), Pictures Depicting Pictures: On the Specification of Visual Languages by Visual Grammars. In Proceedings of IEEE Workshop on Visual Languages, Seattle, Washington, USA, pp. 41–47.

    Google Scholar 

  • Minas, M. and Viehstaedt, G. (1995), DiaGen: A Generator for Diagram Editors Providing Direct Manipulation and Execution of Diagrams. In Proceedings of IEEE Workshop on Visual Languages, Darmstadt, Germany.

    Google Scholar 

  • Nagl, M. (1983), Bibliography on Graph Rewriting Systems (Graph Grammars), Bulletin of European Association of Theoretical Computer Science 20 (Austria), pp. 114–148.

    Google Scholar 

  • Pineda, L. (1992), Reference, Synthesis and Constraint Satisfaction, Eurographics 11, pp. C333–C344.

    Google Scholar 

  • Rekers, J. (1994), On the Use of Graph Grammars forDefining the Syntax of Graphical Languages. In Proceedings of Colloquium on Graph Transformation and its Application in Computer Science, Palma de Mallorca, Spain.

    Google Scholar 

  • Rosenfeld, A. (1990), Array, Tree, and Graph Grammars. In Syntactic and Structural Pattern Recognition: Theory and Applications, Bunke, H. and Sanfeliu, A. (eds.), Singapore: World Scientific.

    Google Scholar 

  • Sablowski, R. and Frick, A. (1997), Automatic Graph Clustering. In Graph Drawing: Symposium on Graph Drawing GD '96, North, S. (ed.), Berkeley, California, USA, Lecture Notes in Computer Science 1190, Springer-Verlag, pp. 4395–400.

    Google Scholar 

  • Shaw, A.C. (1969), A Formal Picture Description Scheme as a Basis for Picture Processing Systems, Information and Control 14, pp. 9–52.

    Article  MATH  Google Scholar 

  • Soiffer, N.M. (1991), The Design of a User Interface for Computer Algebra Systems, Ph.D. Thesis, University of California at Berkeley.

    Google Scholar 

  • Telelogic (1997), http://www.telelogic.se.

    Google Scholar 

  • üsküdarli, S. and Dinesh, T.B. (1995), Towards a visual programming environment generator for algebraic specifications. In IEEE Symposium on Visual Languages, Darmstadt, Germany.

    Google Scholar 

  • Wahlster, W. (1991), User and Discourse Models for Multimodal Communication. In Intelligent User Interfaces, Sullivan, J.W. and Tyler, S.W. (eds.), ACM Press, pp. 45–67.

    Google Scholar 

  • Weimer, D. and Ganapathy, S.K. (1992), Interaction Techniques Using Hand Tracking and Speech Recognition. In Multimedia Interface Design, Blattner, M.H. and Dannenberg, R.B. (eds.), ACM Press, pp. 109–126.

    Google Scholar 

  • Weitzman, L. and Wittenburg, K. (1993), Relational Grammars for Interactive Design. In Proceedings of IEEE Symposium on Visual Languages, Bergen, Norway, pp. 4–11.

    Google Scholar 

  • Weitzman, L., and Wittenburg, K. (1994), Automatic Generation of Multimedia Documents Using Relational Grammars. In Proceedings of ACM Multimedia 94, San Francisco, California, USA, pp. 443–451.

    Google Scholar 

  • Weitzman, L. and Wittenburg, K. (1996), Grammar-based Articulation for Multimedia Document Design, Multimedia Systems 4, PP. 99–111.

    Article  Google Scholar 

  • Wittenburg, K., Weitzman, L., and Talley, J. (1991), Unification-Based Grammars and Tabular Parsing for Graphical Languages, Journal of Visual Languages and Computing 2, pp. 347–370.

    Article  Google Scholar 

  • Wittenburg, K. (1992), Earley-style Parsing for Relational Grammars. In Proceedings of IEEE Workshop on Visual Languages, Seattle, Wa., USA, pp. 192–199.

    Google Scholar 

  • Wittenburg, K. (1996), Predictive Parsing for Unordered Relational Languages. In Recent Advances in Parsing Technologies, Bunt, H. and Tomita, M. (eds.), Kluwer, pp. 389–411.

    Google Scholar 

  • Wittenburg, K. and Weitzman, L. (1997a), Qualitative Visualization of Processes: Attributed Graph Layout and Focusing Techniques. In Graph Drawing: Symposium on Graph Drawing GD '96, North, S. (ed.), Berkeley, California, USA, Lecture Notes in Computer Science 1190, Springer-Verlag, pp. 401–408.

    Google Scholar 

  • Wittenburg, K. and Weitzman, L. (1997b), Relational Grammars: Theory and Practice in a Visual Language Interface for Process Modeling. In Theory of Visual Languages, Lecture Notes in Computer Science, Marriott, K. and Meyer, B. (eds.), Springer Verlag (in press).

    Google Scholar 

  • Wolfram, S. (1996), The Mathematica Book, Cambridge University Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. GTE Laboratories, 40 Sylvan Rd., 02254, Waltham, MA, USA

    Kent Wittenburg

Authors
  1. Kent Wittenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Harry Bunt Robbert-Jan Beun Tijn Borghuis

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag

About this paper

Cite this paper

Wittenburg, K. (1998). Visual language parsing: If I had a hammer.... In: Bunt, H., Beun, RJ., Borghuis, T. (eds) Multimodal Human-Computer Communication. CMC 1995. Lecture Notes in Computer Science, vol 1374. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0052321

Download citation

  • DOI: https://doi.org/10.1007/BFb0052321

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-64380-7

  • Online ISBN: 978-3-540-69764-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation