Abstract
The assessment of intonation, to which intonation verification belongs, has many applications, such as health-impaired people training – from individuals with Parkinson’s disease to children with Autism Spectrum Disorders – and second language learning. Most of the approaches that are found in the literature are based on intensive preprocessing of the audio signal and hand-crafted feature extraction methods, and most of those works do not tackle the particularities of the Portuguese language. In this paper, we present our work on intonation assessment, developed from a database of binarily-labelled Portuguese intonation imitation. This has been done using the set of Low Level Descriptors (LLDs) and eGeMAPS, both extracted with the openSMILE toolkit, and Problem-Agnostic Speech Encoder (PASE) features. We have taken the most informative feature subsets for prosody out of these. Distances between stimulus and imitation – the so-called similarity intonation scores – have been computed applying Dynamic Time Warping (DTW) for different feature subsets, and have afterwards been used as input features of a binary classifier. Performance achieves up to 66.9% of accuracy in the test data set when considering only one feature set, and it increases up to 77.5% for a set of seven features.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arias, J.P., Yoma, N.B., Vivanco, H.: Automatic intonation assessment for computer aided language learning. Speech Commun. 52(3), 254–267 (2010)
Berndt, D.J., Clifford, J.: Using dynamic time warping to find patterns in time series. In: KDD Workshop, Seattle, WA, vol. 10, pp. 359–370 (1994)
Brümmer, N., De Villiers, E.: The BOSARIS toolkit user guide: theory, algorithms and code for binary classifier score processing. Documentation of BOSARIS toolkit, p. 24 (2011)
Cheng, J.: Automatic assessment of prosody in high-stakes English tests. In: Twelfth Annual Conference of the International Speech Communication Association (2011)
Christophe, A., Gout, A., Peperkamp, S., Morgan, J.: Discovering words in the continuous speech stream: the role of prosody. J. Phon. 31(3–4), 585–598 (2003)
Chun, D.M.: Discourse Intonation in L2: From Theory and Research to Practice, vol. 1. John Benjamins Publishing, Amsterdam (2002)
Escudero-Mancebo, D., González-Ferreras, C., Aguilar, L., Estebas-Vilaplana, E., Cardeñoso-Payo, V.: Exploratory use of automatic prosodic labels for the evaluation of Japanese speakers of L2 Spanish (2016)
Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in openSMILE, the Munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM International Conference on Multimedia, MM 2013, pp. 835–838. ACM, New York (2013). https://doi.org/10.1145/2502081.2502224, https://doi.acm.org/10.1145/2502081.2502224
Eyben, F., et al.: The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for voice research and affective computing. IEEE Trans. Affect. Comput. 7(2), 190–202 (2015)
Levitt, A.G.: The acquisition of prosody: evidence from French- and English-learning infants. In: de Boysson-Bardies, B., de Schonen, S., Jusczyk, P., McNeilage, P., Morton, J. (eds.) Developmental Neurocognition: Speech and Face Processing in the First Year of Life. NATO ASI Series (Series D: Behavioural and Social Sciences), vol. 69, pp. 385–398. Springer, Dordrecht (1993). https://doi.org/10.1007/978-94-015-8234-6_31
Li, K., Wu, X., Meng, H.: Intonation classification for L2 English speech using multi-distribution deep neural networks. Comput. Speech Lang. 43, 18–33 (2017)
Ma, M., Evanini, K., Loukina, A., Wang, X., Zechner, K.: Using F0 contours to assess nativeness in a sentence repeat task. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)
Neto, J.P., Martins, C.A., Meinedo, H., Almeida, L.B.: The design of a large vocabulary speech corpus for Portuguese. In: Fifth European Conference on Speech Communication and Technology (1997)
Pascual, S., Ravanelli, M., Serrà, J., Bonafonte, A., Bengio, Y.: Learning problem-agnostic speech representations from multiple self-supervised tasks. arXiv preprint arXiv:1904.03416 (2019)
Paul, R., Shriberg, L.D., McSweeny, J., Cicchetti, D., Klin, A., Volkmar, F.: Brief report: relations between prosodic performance and communication and socialization ratings in high functioning speakers with autism spectrum disorders. J. Autism Dev. Disord. 35(6), 861 (2005)
Povey, D., et al.: The Kaldi speech recognition toolkit. In: IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society (2011)
Pudil, P., Novovičová, J., Kittler, J.: Floating search methods in feature selection. Pattern Recogn. Lett. 15(11), 1119–1125 (1994)
Schuller, B., et al.: The INTERSPEECH 2016 computational paralinguistics challenge: deception, sincerity and native language. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 2001–2005, September 2016. https://doi.org/10.21437/Interspeech.2016-129
da Silva Sousa, M.S.: Prosodic exercises for children with ASD via virtual therapy. Master’s thesis, Instituto Superior Técnico, Lisboa, Portugal (2017)
Truong, Q.T., Kato, T., Yamamoto, S.: Automatic assessment of L2 English word prosody using weighted distances of F0 and intensity contours. In: Interspeech, pp. 2186–2190 (2018)
Acknowledgements
This work was supported by national funds through Fundação para a Ciência e a Tecnologia (FCT), with reference UIDB/50021/2020, through PhD grant SFRH/BD/139473/2018, and project CMUP-ERI/TIC/0033/2014.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Julião, M., Abad, A., Moniz, H. (2020). Comparison of Heterogeneous Feature Sets for Intonation Verification. In: Quaresma, P., Vieira, R., Aluísio, S., Moniz, H., Batista, F., Gonçalves, T. (eds) Computational Processing of the Portuguese Language. PROPOR 2020. Lecture Notes in Computer Science(), vol 12037. Springer, Cham. https://doi.org/10.1007/978-3-030-41505-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-41505-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41504-4
Online ISBN: 978-3-030-41505-1
eBook Packages: Computer ScienceComputer Science (R0)