Kathryn Schaefer Ziemer
ABSTRACT
Automated text analysis (ATA) has been a widely used tool for determining the sentiment of writing samples. However, it is unclear how ATA compares to human ratings of text when estimating affect. There are costs and benefits associated with each method, and comparing the two approaches will help determine which one provides the most useful and accurate results. This study uses 279 journal entries from individuals with chronic pain in order to estimate the positive and negative affect scores reported directly by participants. We use Lasso to select the features that are most predictive of affect. Our results indicate that the model combining human coders and ATA accounts for the most variance in self-reported positive affect scores, resulting in adjusted R-squared=0.36. For negative affect scores, we obtain a lower adjusted R-squared=0.30 with the combined model, however, ATA results in significantly higher adjusted R-squared=0.27 compared to the model using only human coders, R-squared=0.14. This suggests that utilizing human coders may be the most beneficial when the focus is on positive affect, but automated text analysis may be sufficient when studying negative affect.
- I. Ajzen. The theory of planned behavior. Organizational behavior and human decision processes, 50(2):179--211, 1991.Google Scholar
- I. Ajzen. Nature and operation of attitudes. Annual review of psychology, 52(1):27--58, 2001.Google Scholar
- G. W. Alpers, A. J. Winzelberg, C. Classen, H. Roberts, P. Dev, C. Koopman, and C. B. Taylor. Evaluation of computerized text analysis in an internet breast cancer support group. Computers in Human Behavior, 21(2):361--376, 2005.Google ScholarCross Ref
- J. Alvarez-Conrad, L. A. Zoellner, and E. B. Foa. Linguistic predictors of trauma pathology and physical health. Applied Cognitive Psychology, 15(7):S159--S170, 2001.Google ScholarCross Ref
- K. Bakhtiyari, M. Taghavi, and H. Husain. Hybrid affective computing-keyboard, mouse and touch screen: from review to experiment. Neural Computing and Applications, 26(6):1277--1296, 2015. Google ScholarDigital Library
- E. O. Bantum and J. E. Owen. Evaluating the validity of computerized content analysis programs for identification of emotional expression in cancer narratives. Psychological assessment, 21(1):79, 2009.Google ScholarCross Ref
- C. M. Burton and L. A. King. The health benefits of writing about intensely positive experiences. Journal of research in personality, 38(2):150--163, 2004.Google Scholar
- J. Cohen. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1), 1960.Google Scholar
- J. R. Crawford and J. D. Henry. The positive and negative affect schedule (panas): Construct validity, measurement properties and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 43(3):245--265, 2004.Google ScholarCross Ref
- M. C. Davis, A. J. Zautra, and J. W. Reich. Vulnerability to stress among women in chronic pain from fibromyalgia and osteoarthritis davis et al. stress vulnerability. Annals of Behavioral Medicine, 23(3):215--226, 2001.Google ScholarCross Ref
- R. Feldman. Techniques and applications for sentiment analysis. Communications of the ACM, 56(4):82--89, 2013. Google ScholarDigital Library
- M. Fishbein and I. Ajzen. Belief, attitude, intention and behavior: An introduction to theory and research. 1975.Google Scholar
- J. Frattaroli. Experimental disclosure and its moderators: a meta-analysis. Psychological bulletin, 132(6):823, 2006.Google ScholarCross Ref
- B. L. Fredrickson. The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American psychologist, 56(3):218, 2001.Google ScholarCross Ref
- J. H. Kahn, R. M. Tobin, A. E. Massey, and J. A. Anderson. Measuring emotional expression with the linguistic inquiry and word count. The American journal of psychology, pages 263--286, 2007.Google Scholar
- K. Kercher. Assessing subjective well-being in the old-old the panas as a measure of orthogonal dimensions of positive and negative affect. Research on Aging, 14(2):131--168, 1992.Google ScholarCross Ref
- A. Mackinnon, A. F. Jorm, H. Christensen, A. E. Korten, P. A. Jacomb, and B. Rodgers. A short form of the positive and negative affect schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample. Personality and Individual differences, 27(3):405--416, 1999.Google ScholarCross Ref
- S. Mishra, J. Diesner, J. Byrne, and E. Surbeck. Sentiment analysis with incremental human-in-the-loop learning and lexical resource customization. In Proceedings of the 26th ACM Conference on Hypertext & Social Media, pages 323--325. ACM, 2015. Google ScholarDigital Library
- A. Ortigosa, J. M. Martın, and R. M. Carro. Sentiment analysis in facebook and its application to e-learning. Computers in Human Behavior, 31:527--541, 2014. Google ScholarDigital Library
- J. W. Pennebaker and S. K. Beall. Confronting a traumatic event: toward an understanding of inhibition and disease. Journal of abnormal psychology, 95(3):274, 1986.Google Scholar
- J. W. Pennebaker, C. K. Chung, M. Ireland, A. Gonzales, and R. J. Booth. The development and psychometric properties of liwc2007. UT Faculty/Researcher Works, 2007.Google Scholar
- J. W. Pennebaker, M. E. Francis, and R. J. Booth. Linguistic inquiry and word count: Liwc 2001. Mahway: Lawrence Erlbaum Associates, 71:2001, 2001.Google Scholar
- Y. R. Tausczik and J. W. Pennebaker. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24--54, 2010.Google Scholar
- M. Thelwall, K. Buckley, G. Paltoglou, D. Cai, and A. Kappas. Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology, 61(12):2544--2558, 2010. Google ScholarDigital Library
- R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267--288, 1996.Google ScholarCross Ref
- W. Tov, K. L. Ng, H. Lin, and L. Qiu. Detecting well-being via computerized content analysis of brief diary entries. Psychological assessment, 25(4):1069, 2013.Google ScholarCross Ref
- D. Watson, L. A. Clark, and A. Tellegen. Development and validation of brief measures of positive and negative affect: the panas scales. Journal of personality and social psychology, 54(6):1063, 1988.Google Scholar
Index Terms
- Human vs. Automated Text Analysis: Estimating Positive and Negative Affect
Recommendations
Sochiatrist: Signals of Affect in Messaging Data
CSCWMessaging is a common mode of communication, with conversations written informally between individuals. Interpreting emotional affect from messaging data can lead to a powerful form of reflection or act as a support for clinical therapy. Existing ...
Using text to predict psychological and physical health
Given the wide-spread use of social media, text analysis has emerged as a promising way to gather information about individuals. However, it is still unclear which method of text analysis is best for determining different types of information. This ...
Insights from an expressive writing intervention on Facebook to help alleviate depressive symptoms
Depression is one of the most common mental disorders. Studies have shown that various online social network (OSN) activities can be used as a marker for people's moods as well as symptoms of subsyndromal depression, suggesting possibilities for online-...
Comments