Abstract
The multimodal HuComTech corpus aims at annotating, studying and publishing data related to a wide spectrum of markers of human behavior in human-human spoken dialogues. By doing so the final goal is to both understand human cognitive behavior in conversational settings and contribute to the enhancement of human-machine interaction systems. One of the main issues still leaving wide spaces for further development is related to speech prosody, the understanding of its association with possible cognitive processes for the expression of emotions as well as the online production of speech utterances. Since the latter often results in incomplete structures, the study of the relation between grammatical incompleteness and prosody can both contribute to a better understanding of human cognition and the enhancement of cognitive infocommunication systems. The data and analyses presented in this paper are intended to serve both these purposes. Two different approaches will be presented as methods of data exploration: the study of static temporal alignments within the ELAN annotation tool, and the discovery of dynamic temporal patterns using the Theme framework.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Sagisaka Y, Campbell N, Higuchi N (eds) (1996) Computing prosody: computational models for processing spontaneous speech. Springer, New York
Rajeswari KC, Uma Maheswari P (2012) Prosody modeling techniques for text-to-speech synthesis systems—A survey. Int J Comput Appl (0975–8887) 39(16):8
Teixeira JP (2012) Prosody generation model for TTS systems: segmental durations and F0 contours with fujisaki model. LAP LAMBERT Academic Publishing
Chaloupka Z, Hork P (2012) Prosody modelling for TTS systems using statistical methods. In: Cognitive behavioural systems, COST 2102 International training school, Dresden, Germany, February 21–26, 2011. Revised Selected Papers, Springer, Heidelberg, pp 174–183
Roy BC, Frank MC, Roy D (2012) Relating activity contexts to early word learning in dense longitudinal data. In: Proceedings of the 34th annual meeting of the cognitive science society. Sapporo, 2012
Baranyi P, Csapo A (2012) Definition and synergies of cognitive infocommunications. Acta Polytech Hung 9(1):67–83
Sallai G (2012) Defining infocommunications and related terms. Acta Polytech Hung 9(6):5–15
Baranyi P, Csapo A, Varlaki P (2014) An overview of research trends in coginfocom. In: IEEE International conference on intelligent engineering systems, Tihany, pp 181–186
Hunyadi L (2011) Multimodal human-computer interaction technologies. Theoretical modeling and application in speech processing, Argumentum 7, pp 240–260
Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto
Hunyadi L, Incompleteness and fragmentation in spoken language syntax and its relation to prosody and gesturing: cognitive processes versus possible formal cues. Knowledge-based information systems in practice. Springer (to appear)
Szekrnyes I (2014) Annotation and interpretation of prosodic data in the HuComTech corpus for multimodal user interfaces. J Multimodal User Interfaces 8(2):143–150
Magnusson MS (1996) Hidden real-time patterns in intra- and inter-individual behavior: description and detection. Eur J Psychol Assess 12(2):112–123
Ladd DR (1996) Intonational phonology. Cambridge University Press, Cambridge
Edlund J, Heldner M, Hirschberg J (2009) Pause and gap length in face-to-face interaction. In: Proceedings of Interspeech 2009, Brighton
Hunyadi L (2010) Cognitive grouping and recursion in prosody. In: Hulst H van der (ed) Recursion and human language, de Guyter, Berlin & New York, pp 343–370
Hunyadi L (2002) Hungarian sentence prosody and universal grammar. Peter Lang, New York
Abuczki A (2011) A multimodal analysis of the sequential organization of verbal and nonverbal interaction. Argumentum 7:261–279
Hunyadi L (2010) Cognitive grouping and recursion in prosody. In: Hulst, Harry van der (ed) Recursion and human language. Studies in Generative Grammar [SGG] 104. de Gruyter Mouton, pp 343–370
Szekrenyes I (2015) ProsoTool, a method for automatic annotation of fundamental frequency. In: Cognitive Infocommunications (CogInfoCom), 2015 6th IEEE International Conference on, 19–21 Oct. 2015, Györ, IEEE 2015, pp 291–296
Acknowledgments
Research presented in this paper was partly supported by project TÁMOP 4.2.2-C/11/1/KONV-2012 and OTKA NK116402. The work was made possible by the generous support of NeDiMAH (Network for Digital Methods in the Arts and Humanities), a cross-European project of the European Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Hunyadi, L., Szekrényes, I., Kiss, H. (2016). Prosody Enhances Cognitive Infocommunication: Materials from the HuComTech Corpus. In: Esposito, A., Jain, L. (eds) Toward Robotic Socially Believable Behaving Systems - Volume I . Intelligent Systems Reference Library, vol 105. Springer, Cham. https://doi.org/10.1007/978-3-319-31056-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-31056-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31055-8
Online ISBN: 978-3-319-31056-5
eBook Packages: EngineeringEngineering (R0)