Skip to main content
Springer Nature Link
Log in

Imbalanced Stance Detection by Combining Neural and External Features

  • Conference paper
  • First Online:
Statistical Language and Speech Processing (SLSP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11816))

Included in the following conference series:

  • 891 Accesses

Abstract

Stance detection is the task of determining the perspective “or stance” of pairs of text. Classifying the stance (e.g. agree, disagree, discuss or unrelated) expressed in news articles with respect to a certain claim is an important step in detecting fake news. Many neural and traditional models predict well on unrelated and discuss classes while they poorly perform on other minority represented classes in the Fake News Challenge-1 (FNC-1) dataset. We present a simple neural model that combines similarity and statistical features through a MLP network for news-stance detection. Aiding augmented training instances to overcome the data imbalance problem and adding batch-normalization and gaussian-noise layers enable the model to prevent overfitting and improve class-wise and overall accuracy. We also conduct additional experiments with a light-GBM and MLP network using the same features and text augmentation to show their effectiveness. In addition, we evaluate the proposed model on the Argument Reasoning Comprehension (ARC) dataset to assess the generalizability of the model. The experimental results of our models outperform the current state-of-the-art.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://en.wiktionary.org/wiki/Category:English_hedges.

References

  1. Attardi, G., Carta, A., Errica, F., Madotto, A., Pannitto, L.: Fa3l at semeval-2017 task 3: a three embeddings recurrent neural network for question answering. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 299–304 (2017)

    Google Scholar 

  2. Baird, S., Sibley, D., Pan, Y.: Talos targets disinformation with fake news challenge (2017). https://blog.talosintelligence.com/2017/06/talos-fake-news-challenge.html

  3. Baldwin, T., Liang, H., Salehi, B., Hoogeveen, D., Li, Y., Duong, L.: UniMelb at semeval-2016 task 3: identifying similar questions by combining a CNN with string similarity measures. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 851–856 (2016)

    Google Scholar 

  4. Bhatt, G., Sharma, A., Sharma, S., Nagpal, A., Raman, B., Mittal, A.: Combining neural, statistical and external features for fake news stance identification. In: Companion Proceedings of the Web Conference 2018, pp. 1353–1357. International World Wide Web Conferences Steering Committee (2018)

    Google Scholar 

  5. Bogdanova, D., Foster, J., Dzendzik, D., Liu, Q.: If you can’t beat them join them: handcrafted features complement neural nets for non-factoid answer reranking. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, pp. 121–131. Long Papers (2017)

    Google Scholar 

  6. Borges, L., Martins, B., Calado, P.: Combining similarity features and deep representation learning for stance detection in the context of checking fake news. J. Data Inf. Qual. (JDIQ) 11, 14 (2019). ACM

    Article  Google Scholar 

  7. Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)

  8. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)

  9. Conforti, C., Pilehvar, M.T., Collier, N.: Towards automatic fake news detection: cross-level stance detection in news articles. In: Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pp. 40–49 (2018)

    Google Scholar 

  10. Ferreira, W., Vlachos, A.: Emergent: a novel data-set for stance classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1163–1168 (2016)

    Google Scholar 

  11. Ghanem, B., Rosso, P., Rangel, F.: Stance detection in fake news a combined feature representation. In: Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pp. 66–71 (2018)

    Google Scholar 

  12. Habernal, I., Wachsmuth, H., Gurevych, I., Stein, B.: The argument reasoning comprehension task: Identification and reconstruction of implicit warrants. arXiv preprint arXiv:1708.01425 (2017)

  13. Hanselowski, A., et al.: A retrospective analysis of the fake news challenge stance-detection task. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 1859–1874 (2018)

    Google Scholar 

  14. Hanselowski, A., PVS, A., Schiller, B., Caspelherr, F.: Description of the system developed by team Athene in the FNC-1 (2017). https://medium.com/@andre134679/team-athene-on-the-fake-news-challenge-28a5cf5e017b

  15. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). MIT Press

    Article  Google Scholar 

  16. Hutto, C.J., Gilbert, E.: Vader: a parsimonious rule-based model for sentiment analysis of social media text. In: Eighth International AAAI Conference on Weblogs and Social Media (2014)

    Google Scholar 

  17. Kusner, M., Sun, Y., Kolkin, N., Weinberger, K.: From word embeddings to document distances. In: International Conference on Machine Learning, pp. 957–966 (2015)

    Google Scholar 

  18. Masood, R., Aker, A.: The fake news challenge: stance detection using traditional machine learning approaches. In: Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (KMIS), pp. 128–135 (2018)

    Google Scholar 

  19. Mikolov, T., Le, Q.V., Sutskever, I.: Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168 (2013)

  20. Mohtarami, M., Baly, R., Glass, J., Nakov, P., Màrquez, L., Moschitti, A.: Automatic stance detection using end-to-end memory networks. arXiv preprint arXiv:1804.07581 (2018)

  21. Pennington, J., Socher, R., Manning, C.: Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  22. Pomerleau, D., Rao, D.: Fake News Challenge (2017). http://www.fakenewschallenge.org/

  23. Riedel, B., Augenstein, I., Spithourakis, G.P., Riedel, S.: A simple but tough-to-beat baseline for the fake news challenge stance detection task. arXiv preprint arXiv:1707.03264 (2017)

  24. Rossiello, G., Basile, P., Semeraro, G.: Centroid-based text summarization through compositionality of word embeddings. In: Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres, pp. 12–21 (2017)

    Google Scholar 

  25. Shu, K., Sliva, A., Wang, S., Tang, J., Liu, H.: Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor. Newsl. 19(1), 22–36 (2017). ACM

    Article  Google Scholar 

  26. Staiano, J., Guerini, M.: Depechemood: a lexicon for emotion analysis from crowd-annotated news. arXiv preprint arXiv:1405.1605 (2014)

  27. Thorne, J., Chen, M., Myrianthous, G., Pu, J., Wang, X., Vlachos, A.: Fake news stance detection using stacked ensemble of classifiers. In: Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, pp. 80–83 (2017)

    Google Scholar 

  28. Tommasel, A., Rodriguez, J.M., Godoy, D.: Textual aggression detection through deep learning. In: Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 177–187 (2018)

    Google Scholar 

  29. Xu, B., Mohtarami, M., Glass, J.: Adversarial domain adaptation for stance detection. arXiv preprint arXiv:1902.02401 (2019)

  30. Zhang, D., Yang, Z.: Word embedding perturbation for sentence classification. arXiv preprint arXiv:1804.08166 (2018)

  31. Zhang, X., LeCun, Y.: Text understanding from scratch. arXiv preprint arXiv:1502.01710 (2015)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University of Birmingham, Birmingham, UK

    Fuad Mire Hassan & Mark Lee

  2. Faculty of Computing, Simad University, Mogadishu, Somalia

    Fuad Mire Hassan

Authors
  1. Fuad Mire Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuad Mire Hassan .

Editor information

Editors and Affiliations

  1. Rovira i Virgili University, Tarragona, Spain

    Carlos Martín-Vide

  2. Queen Mary University of London, London, UK

    Matthew Purver

  3. Jožef Stefan Institute, Ljubljana, Slovenia

    Senja Pollak

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hassan, F.M., Lee, M. (2019). Imbalanced Stance Detection by Combining Neural and External Features. In: Martín-Vide, C., Purver, M., Pollak, S. (eds) Statistical Language and Speech Processing. SLSP 2019. Lecture Notes in Computer Science(), vol 11816. Springer, Cham. https://doi.org/10.1007/978-3-030-31372-2_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31372-2_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31371-5

  • Online ISBN: 978-3-030-31372-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics