Skip to main content
SpringerLink
Log in

Sentic LSTM: a Hybrid Network for Targeted Aspect-Based Sentiment Analysis

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Sentiment analysis has emerged as one of the most popular natural language processing (NLP) tasks in recent years. A classic setting of the task mainly involves classifying the overall sentiment polarity of the inputs. However, it is based on the assumption that the sentiment expressed in a sentence is unified and consistent, which does not hold in the reality. As a fine-grained alternative of the task, analyzing the sentiment towards a specific target and aspect has drawn much attention from the community for its more practical assumption that sentiment is dependent on a particular set of aspects and entities. Recently, deep neural models have achieved great successes on sentiment analysis. As a functional simulation of the behavior of human brains and one of the most successful deep neural models for sequential data, long short-term memory (LSTM) networks are excellent in learning implicit knowledge from data. However, it is impossible for LSTM to acquire explicit knowledge such as commonsense facts from the training data for accomplishing their specific tasks. On the other hand, emerging knowledge bases have brought a variety of knowledge resources to our attention, and it has been acknowledged that incorporating the background knowledge is an important add-on for many NLP tasks. In this paper, we propose a knowledge-rich solution to targeted aspect-based sentiment analysis with a specific focus on leveraging commonsense knowledge in the deep neural sequential model. To explicitly model the inference of the dependent sentiment, we augment the LSTM with a stacked attention mechanism consisting of attention models for the target level and sentence level, respectively. In order to explicitly integrate the explicit knowledge with implicit knowledge, we propose an extension of LSTM, termed Sentic LSTM. The extended LSTM cell includes a separate output gate that interpolates the token-level memory and the concept-level input. In addition, we propose an extension of Sentic LSTM by creating a hybrid of the LSTM and a recurrent additive network that simulates sentic patterns. In this paper, we are mainly concerned with a joint task combining the target-dependent aspect detection and targeted aspect-based polarity classification. The performance of proposed methods on this joint task is evaluated on two benchmark datasets. The experiment shows that the combination of proposed attention architecture and knowledge-embedded LSTM could outperform state-of-the-art methods in two targeted aspect sentiment tasks. We present a knowledge-rich solution for the task of targeted aspect-based sentiment analysis. Our model can effectively incorporate the commonsense knowledge into the deep neural network and be trained in an end-to-end manner. We show that the two-step attentive neural architecture as well as the proposed Sentic LSTM and H-Sentic-LSTM can achieve an improved performance on resolving the aspect categories and sentiment polarity for a targeted entity in its context over state-of-the-art systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The SentiHood dataset has masked all targeted entities with “Location” + INDEX.

  2. http://github.com/senticnet/concept-parser

  3. http://sentic.net/downloads

  4. On SemEval 2015, we use “Negative,” “Positive,” “Neutral,” and “None.”

  5. http://yelp.com.sg/dataset/challenge

References

  1. Cambria E, Das D, Bandyopadhyay S, Feraco A. A practical guide to sentiment analysis. Cham: Springer; 2017.

    Book  Google Scholar 

  2. Poria S, Cambria E, Bajpai R, Hussain A. A review of affective computing: from unimodal analysis to multimodal fusion. Inf Fusion 2017;37:98–125.

    Article  Google Scholar 

  3. Poria S, Cambria E, Hazarika D, Mazumder N, Zadeh A, Morency L.-P. Context-dependent sentiment analysis in user-generated videos. ACL; 2017. p. 873–83.

  4. Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E. Bayesian network based extreme learning machine for subjectivity detection. J. Frankl. Inst 2018;355(4):1780–1797.

    Article  Google Scholar 

  5. Das S R, Chen MY. Yahoo! for amazon: sentiment extraction from small talk on the web. Manag Sci 2007; 53(9):1375–88.

    Article  Google Scholar 

  6. Morinaga S, Yamanishi K, Tateishi K, Fukushima T. Mining product reputations on the web. Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. New York: ACM; 2002. p. 341–9.

  7. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S. Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 27–35.

  8. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra S M, Eryiğit G. Semeval-2016 task 5: aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). San Diego: Association for Computational Linguistics; 2016. p. 19–30.

  9. Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowl-Based Syst 2016;108:42–9.

    Article  Google Scholar 

  10. Xia Y, Cambria E, Hussain A. Aspnet: aspect extraction by bootstrapping generalization and propagation using an aspect network. Cogn Comput 2015;7(2):241–53.

    Article  Google Scholar 

  11. Poria S, Chaturvedi I, Cambria E, Bisio F. Sentic LDA: improving on LDA with semantic similarity for aspect-based sentiment analysis. IJCNN; 2016. p. 4465–73.

  12. Tang D, Qin B, Feng X, Liu T. Effective LSTMs for target-dependent sentiment classification. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka; 2016. p. 3298–307.

  13. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K. Adaptive recursive neural network for target-dependent twitter sentiment classification. Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 2: Short Papers). Baltimore: Association for Computational Linguistics; 2014. p. 49–54.

  14. Wang B, Liakata M, Zubiaga A, Tdparse R. Procter. Multi-target-specific sentiment recognition on twitter. Proceedings of the 15th conference of the European chapter of the association for computational linguistics: volume 1, Long Papers. Valencia: Association for Computational Linguistics; 2017. p. 483–93.

  15. Saeidi M, Bouchard G, Liakata M, Riedel S. Sentihood: targeted aspect based sentiment analysis dataset for urban neighbourhoods. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka: The COLING 2016 Organizing Committee; 2016. p. 1546–56.

  16. Nguyen T H, Shirai K. Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. Proceedings of the 2015 conference on empirical methods in natural language processing. Lisbon: Association for Computational Linguistics; 2015. p. 2509–14.

  17. Wang Y, Huang M, Zhu X, Zhao L. Attention-based LSTM for aspect-level sentiment classification. Proceedings of the 2016 conference on empirical methods in natural language processing. Austin: Association for Computational Linguistics; 2016. p. 606–15.

  18. Tang D, Qin B, Liu T, Aspect level sentiment classification with deep memory network. Proceedings of the 2016 conference on empirical methods in natural language processing. Austin: Association for Computational Linguistics; 2016. p. 214–24.

  19. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9(8):1735–80.

    Article  PubMed  CAS  Google Scholar 

  20. Cambria E, Hussain A, Havasi C, Eckl C. Common sense computing: from the society of mind to digital intuition and beyond. In: Fierrez J, Ortega J, Esposito A, Drygajlo A, Faundez-Zanuy M, editors. Biometric ID management and multimodal communication, volume 5707 of lecture notes in computer science. Berlin: Springer; 2009, pp. 252–9.

  21. Baccianella S, Esuli A, Sebastiani F. Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. LREC. Valletta: European Language Resources Association (ELRA); 2010. p. 2200–2204.

  22. Cambria E, Poria S, Bajpai R, Schuller B. SenticNet 4: a semantic resource for sentiment analysis based on conceptual primitives. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka: The COLING 2016 Organizing Committee; 2016. p. 2666– 2677.

  23. Ratinov L, Roth D. Design challenges and misconceptions in named entity recognition. Proceedings of the thirteenth conference on computational natural language learning. Association for Computational Linguistics; 2009. p. 147–155.

  24. Ma Y, Kim J-J, Bigot B, Khan TM. Feature-enriched word embeddings for named entity recognition in open-domain conversations. 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE; 2016. p. 6055–6059.

  25. Xu Z, Liu B, Wang B, Sun C, Wang X. Incorporating loose-structured knowledge into LSTM with recall gate for conversation modeling. arXiv:1605.05110. 2016.

  26. Li Y, Pan Q, Yang T, Wang S H, Tang J L, Cambria E. Learning word representations for sentiment analysis. Cogn Comput 2017 ;9(6):843–51.

    Article  Google Scholar 

  27. Ofek N, Poria S, Rokach L, Cambria E, Hussain A, Shabtai A. Unsupervised commonsense knowledge enrichment for domain-specific sentiment analysis. Cogn Comput 2016;8(3):467–77.

    Article  Google Scholar 

  28. Yang B, Mitchell T. Leveraging knowledge bases in LSTMs for improving machine reading. Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers). Vancouver: Association for Computational Linguistics; 2017. p. 1436–1446.

  29. Cambria E, Hussain A. Sentic album: content-, concept-, and context-based online personal photo management system. Cogn Comput 2012;4(4):477–96.

    Article  Google Scholar 

  30. Wang Q-F, Cambria E, Liu C-L, Hussain A. Common sense knowledge for handwritten Chinese text recognition. Cogn Comput 2013;5(2):234–42.

    Article  Google Scholar 

  31. Cambria E, Fu J, Bisio F, Poria S. AffectiveSpace 2: enabling affective intuition for concept-level sentiment analysis. AAAI; 2015. p. 508–514.

  32. Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Dcu L. Tounsi. Aspect-based polarity classification for SemEval task 4. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 223–229.

  33. Kiritchenko S, Zhu X, Cherry C, Mohammad S. NRC-Canada-2014: detecting aspects and sentiment in customer reviews. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 437–442.

  34. Lakkaraju H, Socher R, Manning C. Aspect specific sentiment analysis using hierarchical deep learning. NIPS workshop on deep learning and representation learning. Curran Associates Inc.; 2014.

  35. Chen P, Sun Z, Bing L, Yang W. Recurrent attention network on memory for aspect sentiment analysis. Proceedings of the 2017 conference on empirical methods in natural language processing. Copenhagen: Association for Computational Linguistics; 2017. p. 463–472.

  36. Rahman A, Ng V. Conference resolution with world knowledge. Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1. Association for Computational Linguistics; 2011. p. 814–824.

  37. Nakashole N, Mitchell TM. A knowledge-intensive model for prepositional phrase attachment. ACL (1); 2015. p. 365–375.

  38. Ahn S, Choi H, Pärnamaa T., Bengio Y. A neural knowledge language model. arXiv:1608.00318. 2016.

  39. Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Trans Signal Process 1997;45(11): 2673–81.

    Article  Google Scholar 

  40. Oneto L, Bisio F, Cambria E, Anguita D. Semi-supervised learning for affective common-sense reasoning. Cogn Comput 2017;9(1):18–42.

    Article  Google Scholar 

  41. Lee K, Levy O, Zettlemoyer L. Recurrent additive networks. arXiv:1705.07393. 2017.

  42. Poria S, Cambria E, Winterstein G, Huang G-B. Sentic patterns Dependency-based rules for concept-level sentiment analysis. Knowl-Based Syst 2014;69:45–63.

    Article  Google Scholar 

  43. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. SemEval-2015 task 12: aspect based sentiment analysis. Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). Denver: Association for Computational Linguistics; 2015. p. 486–495.

  44. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems; 2013. p. 3111–3119.

  45. He R, McAuley J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. Proceedings of the 25th international conference on world wide web. International World Wide Web Conferences Steering Committee; 2016. p. 507–517.

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanyang Technological University, 50 Nanyang Ave, Singapore, 639798, Singapore

    Yukun Ma, Haiyun Peng & Erik Cambria

  2. Alef Education Consultancy, Abu Dhabi, United Arab Emirates

    Tahir Khan

  3. School of Natural Sciences, University of Stirling, Stirling, Scotland, FK9 4LA, UK

    Amir Hussain

Authors
  1. Yukun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiyun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tahir Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amir Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik Cambria.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was not required as no human or animals were involved.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, Y., Peng, H., Khan, T. et al. Sentic LSTM: a Hybrid Network for Targeted Aspect-Based Sentiment Analysis. Cogn Comput 10, 639–650 (2018). https://doi.org/10.1007/s12559-018-9549-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-018-9549-x

Keywords

Search

Navigation