Skip to main content
SpringerLink
Log in

Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformers

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13980))

Included in the following conference series:

Abstract

The analysis of emotions expressed in text has numerous applications. In contrast to categorical analysis, focused on classifying emotions according to a pre-defined set of common classes, dimensional approaches can offer a more nuanced way to distinguish between different emotions. Still, dimensional methods have been less studied in the literature. Considering a valence-arousal dimensional space, this work assesses the use of pre-trained Transformers to predict these two dimensions on a continuous scale, with input texts from multiple languages and domains. We specifically combined multiple annotated datasets from previous studies, corresponding to either emotional lexica or short text documents, and evaluated models of multiple sizes and trained under different settings. Our results show that model size can have a significant impact on the quality of predictions, and that by fine-tuning a large model we can confidently predict valence and arousal in multiple languages. We make available the code, models, and supporting data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.github.com/gmendes9/multilingual_va_prediction.

  2. 2.

    https://www.cia.gov/the-world-factbook/countries/world/people-and-society.

  3. 3.

    https://www.ethnologue.com/.

  4. 4.

    https://mapie.readthedocs.io/en/latest/.

  5. 5.

    https://huggingface.co/laion/CLIP-ViT-H-14-frozen-xlm-roberta-large-laion5B-s13B-b90k.

References

  1. Alm, C.O., Roth, D., Sproat, R.: Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2005)

    Google Scholar 

  2. Angelopoulos, A.N., Bates, S.: A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 (2021)

  3. Barron, J.T.: A general and adaptive robust loss function. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  4. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  5. Bradley, M.M., Lang, P.J.: Affective Norms for English Words (ANEW): instruction manual and affective ratings. Technical report C-1, The Center for Research in Psychophysiology, University of Florida (1999)

    Google Scholar 

  6. Bradley, M.M., Lang, P.J.: Affective Norms for English Text (ANET): affective ratings of text and instruction manual. Technical report D-1, University of Florida (2007)

    Google Scholar 

  7. Buechel, S., Hahn, U.: Emotion analysis as a regression problem - dimensional models and their implications on emotion representation and metrical evaluation. In: Proceedings of the European Conference on Artificial Intelligence (2016)

    Google Scholar 

  8. Buechel, S., Hahn, U.: EmoBank: studying the impact of annotation perspective and representation format on dimensional emotion analysis. In: Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (2017)

    Google Scholar 

  9. Buechel, S., Hahn, U.: A flexible mapping scheme for discrete and dimensional emotion representations: evidence from textual stimuli. In: Proceedings of the Annual Meeting of the Cognitive Science Society (2017)

    Google Scholar 

  10. Buechel, S., Hahn, U.: Readers vs. writers vs. texts: coping with different perspectives of text understanding in emotion annotation. In: Proceedings of the Linguistic Annotation Workshop (2017)

    Google Scholar 

  11. Buechel, S., Rücker, S., Hahn, U.: Learning and evaluating emotion lexicons for 91 languages. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (2020)

    Google Scholar 

  12. Busso, C., et al.: IEMOCAP: interactive emotional dyadic motion capture database. Lang. Resour. Eval. 42(4), 335–359 (2008)

    Article  Google Scholar 

  13. Carlsson, F., Eisen, P., Rekathati, F., Sahlgren, M.: Cross-lingual and multilingual CLIP. In: Proceedings of the Language Resources and Evaluation Conference (2022)

    Google Scholar 

  14. Carretié, L., Tapia, M., López-Martín, S., Albert, J.: EmoMadrid: an emotional pictures database for affect research. Motiv. Emotion 43(6) (2019)

    Google Scholar 

  15. Citron, F.M.M., Lee, M., Michaelis, N.: Affective and psycholinguistic norms for German conceptual metaphors (COMETA). Behav. Res. Methods 52(3), 1056–1072 (2020)

    Article  Google Scholar 

  16. Citron, F.M., Cacciari, C., Kucharski, M., Beck, L., Conrad, M., Jacobs, A.M.: When emotions are expressed figuratively: psycholinguistic and Affective Norms of 619 Idioms for German (PANIG). Behav. Res. Methods 48(1), 91–111 (2016)

    Article  Google Scholar 

  17. Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (2020)

    Google Scholar 

  18. Ćoso, B., Guasch, M., Ferré, P., Hinojosa, J.A.: Affective and concreteness norms for 3,022 Croatian words. Q. J. Exp. Psychol. 72(9), 2302–2312 (2019)

    Article  Google Scholar 

  19. Dan-Glauser, E.S., Scherer, K.R.: The Geneva Affective Picture Database (GAPED): a new 730-picture database focusing on valence and normative significance. Behav. Res. Methods 43(2) (2011)

    Google Scholar 

  20. Du, S., Zhang, X.: Aicyber’s system for IALP 2016 shared task: character-enhanced word vectors and boosted neural networks. In: Proceedings of the International Conference on Asian Language Processing (2016)

    Google Scholar 

  21. Eilola, T.M., Havelka, J.: Affective norms for 210 British English and Finnish nouns. Behav. Res. Methods 42(1), 134–140 (2010)

    Article  Google Scholar 

  22. Ekman, P.: An argument for basic emotions. Cogn. Emotion 6(3–4), 169–200 (1992)

    Article  Google Scholar 

  23. Fairfield, B., Ambrosini, E., Mammarella, N., Montefinese, M.: Affective norms for Italian words in older adults: age differences in ratings of valence, arousal and dominance. PLoS ONE 12(1), e0169472 (2017)

    Article  Google Scholar 

  24. Francisco, V., Hervás, R., Peinado, F., Gervás, P.: EmoTales: creating a corpus of folk tales with emotional annotations. Lang. Resour. Eval. 46(3), 341–381 (2012)

    Article  Google Scholar 

  25. Gilet, A.L., Grühn, D., Studer, J., Labouvie-Vief, G.: Valence, arousal, and imagery ratings for 835 French attributes by young, middle-aged, and older adults: the French Emotional Evaluation List (FEEL). Eur. Rev. Appl. Psychol. 62(3), 173–181 (2012)

    Article  Google Scholar 

  26. Hollis, G., Westbury, C., Lefsrud, L.: Extrapolating human judgments from skip-gram vector representations of word meaning. Q. J. Exp. Psychol. 70, 1–45 (2016)

    Google Scholar 

  27. Imbir, K.K.: Affective Norms for 4900 Polish Words Reload (ANPW_R): assessments for valence, arousal, dominance, origin, significance, concreteness, imageability and age of acquisition. Front. Psychol. 7, 1081 (2016)

    Article  Google Scholar 

  28. Imbir, K.K.: The Affective Norms for Polish Short Texts (ANPST) database properties and impact of participants’ population and sex on affective ratings. Front. Psychol. 8, 855 (2017)

    Article  Google Scholar 

  29. Kanske, P., Kotz, S.A.: Leipzig affective norms for German: a reliability study. Behav. Res. Methods 42(4), 987–991 (2010)

    Article  Google Scholar 

  30. Kapucu, A., Kılıç, A., Özkılıç, Y., Sarıbaz, B.: Turkish emotional word norms for arousal, valence, and discrete emotion categories. Psychol. Rep. 124(1), 188–209 (2021)

    Article  Google Scholar 

  31. Kim, H.R., Kim, Y.S., Kim, S.J., Lee, I.K.: Building emotional machines: recognizing image emotions through deep neural networks. IEEE Trans. Multimedia 20(11) (2018)

    Google Scholar 

  32. Kratzwald, B., Ilić, S., Kraus, M., Feuerriegel, S., Prendinger, H.: Deep learning for affective computing: text-based emotion recognition in decision support. Decis. Support Syst. 115, 24–35 (2018)

    Article  Google Scholar 

  33. Kron, A., Goldstein, A., Lee, D., Gardhouse, K., Anderson, A.: How are you feeling? Revisiting the quantification of emotional qualia. Psychol. Sci. 24(8), 1503–1511 (2013)

    Article  Google Scholar 

  34. Kurdi, B., Lozano, S., Banaji, M.R.: Introducing the open affective standardized image set (OASIS). Behav. Res. Methods 49(2) (2017)

    Google Scholar 

  35. Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International Affective Picture System (IAPS): affective ratings of pictures and instruction manual. Technical report, NIMH Center for the Study of Emotion and Attention (2005)

    Google Scholar 

  36. Lee, L.H., Li, J.H., Yu, L.C.: Chinese EmoBank: building valence-arousal resources for dimensional sentiment analysis. Trans. Asian Low-Resource Lang. Inf. Process. 21(4), 1–18 (2022)

    Article  Google Scholar 

  37. Marchewka, A., Żurawski, Ł., Jednoróg, K., Grabowska, A.: The Nencki Affective Picture System (NAPS): introduction to a novel, standardized, wide-range, high-quality, realistic picture database. Behav. Res. Methods 46(2) (2014)

    Google Scholar 

  38. Mohammad, S.: Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (2018)

    Google Scholar 

  39. Monnier, C., Syssau, A.: Affective Norms for French words (FAN). Behav. Res. Methods 46(4), 1128–1137 (2014)

    Article  Google Scholar 

  40. Moors, A., et al.: Norms of valence, arousal, dominance, and age of acquisition for 4,300 Dutch words. Behav. Res. Methods 45(1), 169–177 (2013)

    Article  Google Scholar 

  41. Paltoglou, G., Theunis, M., Kappas, A., Thelwall, M.: Predicting emotional responses to long informal text. IEEE Trans. Affect. Comput. 4(1), 106–115 (2013)

    Article  Google Scholar 

  42. Pinheiro, A.P., Dias, M., Pedrosa, J., Soares, A.P.: Minho Affective Sentences (MAS): probing the roles of sex, mood, and empathy in affective ratings of verbal stimuli. Behav. Res. Methods 49(2), 698–716 (2017)

    Article  Google Scholar 

  43. Pires, T., Schlinger, E., Garrette, D.: How multilingual is multilingual BERT? In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (2019)

    Google Scholar 

  44. Preoţiuc-Pietro, D., et al.: Modelling valence and arousal in Facebook posts. In: Proceedings of the Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (2016)

    Google Scholar 

  45. Recchia, G., Louwerse, M.: Reproducing affective norms with lexical co-occurrence statistics: predicting valence, arousal, and dominance. Qu. J. Exp. Psychol. 68, 1–41 (2014)

    Google Scholar 

  46. Riegel, M., et al.: Nencki Affective Word List (NAWL): the cultural adaptation of the Berlin Affective Word List-Reloaded (BAWL-R) for Polish. Behav. Res. Methods 47(4), 1222–1236 (2015)

    Article  Google Scholar 

  47. Russell, J.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980)

    Article  Google Scholar 

  48. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

  49. Schmidtke, D.S., Schröder, T., Jacobs, A.M., Conrad, M.: ANGST: affective norms for German sentiment terms derived from the affective norms for English words. Behav. Res. Methods 46(4), 1108–1118 (2014)

    Article  Google Scholar 

  50. Scott, G.G., Keitel, A., Becirspahic, M., Yao, B., Sereno, S.C.: The Glasgow norms: ratings of 5,500 words on nine scales. Behav. Res. Methods 51(3), 1258–1270 (2019)

    Article  Google Scholar 

  51. Sedoc, J., Preoţiuc-Pietro, D., Ungar, L.: Predicting emotional word ratings using distributional representations and signed clustering. In: Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (2017)

    Google Scholar 

  52. Shad Akhtar, M., Ghosal, D., Ekbal, A., Bhattacharyya, P., Kurohashi, S.: A multi-task ensemble framework for emotion, sentiment and intensity prediction. arXiv preprint arXiv:1808.01216 (2018)

  53. Soares, A., Comesaña, M., Pinheiro, A., Simões, A., Frade, S.: The adaptation of the Affective Norms for English Words (ANEW) for European Portuguese. Behav. Res. Methods 44(1), 256–269 (2012)

    Article  Google Scholar 

  54. Söderholm, C., Häyry, E., Laine, M., Karrasch, M.: Valence and arousal ratings for 420 Finnish nouns by age and gender. PLoS ONE 8(8), e72859 (2013)

    Article  Google Scholar 

  55. Stadthagen-Gonzalez, H., Imbault, C., Pérez Sánchez, M.A., Brysbaert, M.: Norms of valence and arousal for 14,031 Spanish words. Behav. Res. Methods 49(1), 111–123 (2017)

    Article  Google Scholar 

  56. Tris Atmaja, B., Akagi, M.: Evaluation of error- and correlation-based loss functions for multitask learning dimensional speech emotion recognition. J. Phys: Conf. Ser. 1896, 012004 (2021)

    Google Scholar 

  57. Verheyen, S., De Deyne, S., Linsen, S., Storms, G.: Lexicosemantic, affective, and distributional norms for 1,000 Dutch adjectives. Behav. Res. Methods 52(3), 1108–1121 (2020)

    Article  Google Scholar 

  58. Vo, M.L.H., Conrad, M., Kuchinke, L., Urton, K., Hofmann, M.J., Jacobs, A.M.: The Berlin Affective Word List Reloaded (BAWL-R). Behav. Res. Methods 41(2), 534–538 (2009)

    Article  Google Scholar 

  59. Wang, J., Yu, L.C., Lai, K., Zhang, X.: Tree-structured regional CNN-LSTM model for dimensional sentiment analysis. IEEE/ACM Trans. Audio Speech Lang. Process. 28, 581–591 (2020)

    Article  Google Scholar 

  60. Warriner, A.B., Kuperman, V., Brysbaert, M.: Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav. Res. Methods 45(4), 1191–1207 (2013)

    Article  Google Scholar 

  61. Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2020)

    Google Scholar 

  62. Wu, C., Wu, F., Huang, Y., Wu, S., Yuan, Z.: THU_NGN at IJCNLP-2017 task 2: dimensional sentiment analysis for Chinese phrases with deep LSTM. In: Proceedings of the International Joint Conference on Natural Language Processing (2017)

    Google Scholar 

  63. Xie, H., Lin, W., Lin, S., Wang, J., Yu, L.C.: A multi-dimensional relation model for dimensional sentiment analysis. Inf. Sci. 579, 832–844 (2021)

    Article  MathSciNet  Google Scholar 

  64. Xu, X., Li, J., Chen, H.: Valence and arousal ratings for 11,310 simplified Chinese words. Behav. Res. Methods 54(1), 26–41 (2022)

    Article  Google Scholar 

  65. Yee, L.: Valence, arousal, familiarity, concreteness, and imageability ratings for 292 two-character Chinese nouns in Cantonese speakers in Hong Kong. PLoS ONE 12(3), e0174569 (2017)

    Article  MathSciNet  Google Scholar 

  66. Yu, L.C., et al.: Building Chinese affective resources in valence-arousal dimensions. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (2016)

    Google Scholar 

  67. Yu, L.C., Wang, J., Lai, K.R., Zhang, X.: Pipelined neural networks for phrase-level sentiment intensity prediction. IEEE Trans. Affect. Comput. 11(3), 447–458 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the European Union’s H2020 research and innovation programme, under grant agreement No. 874850 (MOOD), as well as by the Portuguese Recovery and Resilience Plan (RRP) through project C645008882-00000055 (Responsible.AI), and by Fundação para a Ciência e Tecnologia (FCT), through the INESC-ID multi-annual funding with reference UIDB/50021/2020, and through the projects with references DSAIPA/DS/0102/2019 (DEBAQI) and PTDC/CCI-CIF/32607/2017 (MIMU).

Author information

Authors and Affiliations

  1. Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal

    Gonçalo Azevedo Mendes & Bruno Martins

  2. INESC-ID, Lisbon, Portugal

    Gonçalo Azevedo Mendes & Bruno Martins

Authors
  1. Gonçalo Azevedo Mendes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruno Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno Martins .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Jaap Kamps

  2. Université Grenoble-Alpes, Saint-Martin-d’Hères, France

    Lorraine Goeuriot

  3. Università della Svizzera Italiana, Lugano, Switzerland

    Fabio Crestani

  4. University of Copenhagen, Copenhagen, Denmark

    Maria Maistro

  5. University of Tsukuba, Ibaraki, Japan

    Hideo Joho

  6. Dublin City University, Dublin, Ireland

    Brian Davis

  7. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  8. Universität Regensburg, Regensburg, Germany

    Udo Kruschwitz

  9. Dublin City University, Dublin, Ireland

    Annalina Caputo

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mendes, G.A., Martins, B. (2023). Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformers. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13980. Springer, Cham. https://doi.org/10.1007/978-3-031-28244-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28244-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28243-0

  • Online ISBN: 978-3-031-28244-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation