Abstract
This article gives an overview of the basic dialog shell for multimodal interaction that was created as the foundation for the software components developed in the Core Technology Cluster of the THESEUS research program. It explains the basic setup of systems based on the dialog shell for specific use cases. The general processing paradigm for interaction in our framework using semantic interface elements as well as several dialog phenomena handled by the dialog shell are described. Finally, a selection of different demonstrator systems that were built for a range of applications is presented briefly. They offer access to quite varied use case scenarios featuring multimodal input and output, including gestures, multi-touch, spoken input and output, and interactions across mobile and stationary devices.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Generally, an essential action of any rule is to remove the WME that triggered it, to avoid endless re-firing.
References
J. Alexandersson, T. Becker, Overlay as the basic operation for discourse processing in a multimodal dialogue system, in Proceedings of the IJCAI Workshop Knowledge and Reasoning in Practical Dialogue Systems, Seattle, 2001
S. Bergweiler, M. Deru, D. Porta, Integrating a multitouch Kiosk system with mobile devices and multimodal interaction, in Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS ’10), Saarbrücken (ACM, New York, 2010), pp. 245–246, http://doi.acm.org/10.1145/1936652.1936698
B. Carpenter, The Logic of Typed Feature Structures (Cambridge University Press, Cambridge, UK, 1992)
M.T. Maybury, W. Wahlster, Readings in Intelligent User Interfaces (Morgan Kaufmann Publishers, San Francisco, 1998)
N. Reithinger, S. Bergweiler, R. Engel, G. Herzog, N. Pfleger, M. Romanelli, D. Sonntag, A look under the hood – design and development of the first smartweb system demonstrator, in Proceedings of the 7th International Conference on Multimodal Interfaces (ICMI ’05), Trento (ACM, New York, 2005), pp. 159–166
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Löckelt, M., Deru, M., Schulz, C.H., Bergweiler, S., Becker, T., Reithinger, N. (2014). A Unified Approach for Semantic-Based Multimodal Interaction. In: Wahlster, W., Grallert, HJ., Wess, S., Friedrich, H., Widenka, T. (eds) Towards the Internet of Services: The THESEUS Research Program. Cognitive Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-06755-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-06755-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06754-4
Online ISBN: 978-3-319-06755-1
eBook Packages: Computer ScienceComputer Science (R0)