Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Intelligent Data Analysis

IDA 2016: Advances in Intelligent Data Analysis XV pp 368–380Cite as

Stability Evaluation of Event Detection Techniques for Twitter

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9897))

Abstract

Twitter continues to gain popularity as a source of up-to-date news and information. As a result, numerous event detection techniques have been proposed to cope with the steadily increasing rate and volume of social media data streams. Although most of these works conduct some evaluation of the proposed technique, comparing their effectiveness is a challenging task. In this paper, we examine the challenges to reproducing evaluation results for event detection techniques. We apply several event detection techniques and vary four parameters, namely time window (15 vs. 30 vs. 60 mins), stopwords (include vs. exclude), retweets (include vs. exclude), and the number of terms that define an event (1...5 terms). Our experiments use real-world Twitter streaming data and show that varying these parameters alone significantly influences the outcomes of the event detection techniques, sometimes in unforeseen ways. We conclude that even minor variations in event detection techniques may lead to major difficulties in reproducing experiments.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://dev.twitter.com (April 28, 2016).

References

  1. Aiello, L.M., Petkos, G., Martin, C., Corney, D., Papadopoulos, S., Skraba, R., Göker, A., Kompatsiaris, I.: Sensing trending topics in Twitter. IEEE Trans. Multimedia 15(6), 1268–1282 (2013)

    Article  Google Scholar 

  2. Alvanaki, F., Michel, S., Ramamritham, K., Weikum, G.: See what’s enBlogue: real-time emergent topic identification in social media. In: Proceedings of International Conference on Extending Database Technology (EDBT), pp. 336–347 (2012)

    Google Scholar 

  3. Becker, H., Naaman, M., Gravano, L.: Beyond trending topics: real-world event identification on Twitter. In: Proceedings of International Conference on Weblogs and Social Media (ICWSM), pp. 438–441 (2011)

    Google Scholar 

  4. Beel, J., Breitinger, C., Langer, S., Lommatzsch, A., Gipp, B.: Towards reproducibility in recommender-systems research. User Model. User-Adap. Inter. 26(1), 69–101 (2016)

    Article  Google Scholar 

  5. Beel, J., Langer, S.: A Comparison of offline evaluations, online evaluations, and user studies in the context of research-paper recommender systems. In: Kapidakis, S., Mazurek, C., Werla, M. (eds.) TPDL 2015. LNCS, vol. 9316, pp. 153–168. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24592-8_12

    Chapter  Google Scholar 

  6. Beel, J., Langer, S., Genzmehr, M., Nürnberger, A.: Introducing Docear’s research paper recommender system. In: Proceedings of Joint Conference on Digital Libraries (JCDL), pp. 459–460 (2013)

    Google Scholar 

  7. Bethard, S., Jurafsky, D.: Who should i cite: learning literature search models from citation behavior. In: Proceedings of International Conference on Information and Knowledge Management (CIKM), pp. 609–618 (2010)

    Google Scholar 

  8. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  9. Cordeiro, M.: Twitter event detection: combining wavelet analysis and topic inference summarization. In: Proceedings of Doctoral Symposium on Informatics Engineering (DSIE) (2012)

    Google Scholar 

  10. Farzindar, A., Khreich, W.: A survey of techniques for event detection in Twitter. Comput. Intell. 31(1), 132–164 (2015)

    Article  MathSciNet  Google Scholar 

  11. Guille, A., Favre, C.: Mention-anomaly-based event detection and tracking in Twitter. In: Proceedings of International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 375–382 (2014)

    Google Scholar 

  12. He, Q., Pei, J., Kifer, D., Mitra, P., Giles, L.: Context-aware citation recommendation. In: Proceedings of International Conference on World Wide Web (WWW), pp. 421–430 (2010)

    Google Scholar 

  13. Li, C., Sun, A., Datta, A.: Twevent: segment-based event detection from tweets. In: Proceedings of International Conference on Information and Knowledge Management (CIKM), pp. 155–164 (2012)

    Google Scholar 

  14. Lu, Y., He, J., Shan, D., Yan, H.: Recommending citations with translation model. In: Proceedings of International Conference on Information and Knowledge Management (CIKM), pp. 2017–2020 (2011)

    Google Scholar 

  15. Madani, A., Boussaid, O., Zegour, D.E.: What’s happening: a survey of tweets event detection. In: Proceedings of International Conference on Communications, Computation, Networks and Technologies (INNOV), pp. 16–22 (2014)

    Google Scholar 

  16. Mathioudakis, M., Koudas, N.: TwitterMonitor: trend detection over the Twitter stream. In: Proceedings of International Conference on Management of Data (SIGMOD), pp. 1155–1158 (2010)

    Google Scholar 

  17. McCallum, A.K.: MALLET: A Machine Learning for Language Toolkit (2002). http://mallet.cs.umass.edu

  18. McCreadie, R., Soboroff, I., Lin, J., Macdonald, C., Ounis, I., McCullough, D.: On building a reusable Twitter corpus. In: Proceedings of International Conference on Research and Development in Information Retrieval (SIGIR), pp. 1113–1114 (2012)

    Google Scholar 

  19. McMinn, A.J., Moshfeghi, Y., Jose, J.M.: Building a large-scale corpus for evaluating event detection on Twitter. In: Proceedings of International Conference on Information and Knowledge Management (CIKM), pp. 409–418 (2013)

    Google Scholar 

  20. Nurwidyantoro, A., Winarko, E.: Event detection in social media: a survey. In: Proceedings of International Conference on ICT for Smart Society (ICISS), pp. 1–5 (2013)

    Google Scholar 

  21. Parikh, R., Karlapalem, K.: ET: Events from Tweets. In: Proceedings of International Conference Companion on World Wide Web (WWW), pp. 613–620 (2013)

    Google Scholar 

  22. Petrović, S., Osborne, M., Lavrenko, V.: Streaming first story detection with application to Twitter. In: Proceedings of Conference on the North American Chapter of the Association for Computational Linguistics (HLT), pp. 181–189 (2010)

    Google Scholar 

  23. Petrović, S., Osborne, M., Lavrenko, V.: Using paraphrases for improving first story detection in news and Twitter. In: Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT), pp. 338–346 (2012)

    Google Scholar 

  24. Weiler, A.: Design and evaluation of event detection techniques for social media data streams. Ph.D. thesis, University of Konstanz, Konstanz (2016)

    Google Scholar 

  25. Weiler, A., Grossniklaus, M., Scholl, M.H.: Event identification and tracking in social media streaming data. In: Proceedings of EDBT Workshop on Multimodal Social Data Management (MSDM), pp. 282–287 (2014)

    Google Scholar 

  26. Weiler, A., Grossniklaus, M., Scholl, M.H.: Evaluation measures for event detection techniques on Twitter data streams. In: Maneth, S. (ed.) BICOD 2015. LNCS, vol. 9147, pp. 108–119. Springer, Heidelberg (2015). doi:10.1007/978-3-319-20424-6_11

    Chapter  Google Scholar 

  27. Weiler, A., Grossniklaus, M., Scholl, M.H.: Run-time and task-based performance of event detection techniques for Twitter. In: Zdravkovic, J., Kirikova, M., Johannesson, P. (eds.) CAiSE 2015. LNCS, vol. 9097, pp. 35–49. Springer, Heidelberg (2015). doi:10.1007/978-3-319-19069-3_3

    Chapter  Google Scholar 

  28. Weiler, A., Scholl, M.H., Wanner, F., Rohrdantz, C.: Event identification for local areas using social media streaming data. In: Proceedings of SIGMOD Workshop on Databases and Social Networks (DBSocial), pp. 1–6 (2013)

    Google Scholar 

  29. Weng, J., Lee, B.S.: Event detection in Twitter. In: Proceedings of International Conference on Weblogs and Social Media (ICWSM), pp. 401–408 (2011)

    Google Scholar 

  30. Zarrinkalam, F., Kahani, M.: SemCiR - a citation recommendation system based on a novel semantic distance measure. Program: Electron. Libr. Inf. Syst. 47(1), 92–112 (2013)

    Article  Google Scholar 

Download references

Acknowledgement

The research presented in this paper is funded in part by the Deutsche Forschungsgemeinschaft (DFG), Grant No. GR 4497/4: “Adaptive and Scalable Event Detection Techniques for Twitter Data Streams” and by a fellowship within the FITweltweit programme of the German Academic Exchange Service (DAAD). We would also like to thank the students Christina Papavasileiou, Harry Schilling, and Wai-Lok Cheung for their contributions to the implementations of WATIS, EDCoW, and enBlogue.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Konstanz, 78457, Konstanz, Germany

    Andreas Weiler, Bela Gipp & Michael Grossniklaus

  2. Digital Contents and Media Sciences Research Division, National Institute of Informatics (NII), Tokyo, 101-8430, Japan

    Joeran Beel

Authors
  1. Andreas Weiler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joeran Beel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bela Gipp
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Grossniklaus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joeran Beel .

Editor information

Editors and Affiliations

  1. Stockholm University , Stockholm, Sweden

    Henrik Boström

  2. Leiden University , Leiden, The Netherlands

    Arno Knobbe

  3. University of Porto , Porto, Portugal

    Carlos Soares

  4. Stockholm University , Stockholm, Sweden

    Panagiotis Papapetrou

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Weiler, A., Beel, J., Gipp, B., Grossniklaus, M. (2016). Stability Evaluation of Event Detection Techniques for Twitter. In: Boström, H., Knobbe, A., Soares, C., Papapetrou, P. (eds) Advances in Intelligent Data Analysis XV. IDA 2016. Lecture Notes in Computer Science(), vol 9897. Springer, Cham. https://doi.org/10.1007/978-3-319-46349-0_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46349-0_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46348-3

  • Online ISBN: 978-3-319-46349-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation