Skip to main content
SpringerLink
Log in

NDNET: A Unified Framework for Anomaly and Novelty Detection

  • Conference paper
  • First Online:
Architecture of Computing Systems (ARCS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13642))

Included in the following conference series:

Abstract

We introduce NDNET (https://novelty-detection.net/p/ndnet), an anomaly and novelty detection library that implements various detection algorithms adjusted for online processing of data streams. The intention of this library is threefold: 1) Make experimentation with different anomaly and novelty detection algorithms simple. 2) Support the development of new novelty detection approaches by providing the mCANDIES framework. 3) Provide fundamentals to analyze and evaluate novelty detection algorithms on data streams. The library is freely available and developed as open-source software.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://novelty-detection.net/p/ndnet.

References

  1. Pimentel, M.A., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Signal Process. 99, 215–249 (2014)

    Article  Google Scholar 

  2. de Faria, E.R., de Leon Ferreira, A.C.P., Gama, J.: MINAS: multiclass learning algorithm for novelty detection in data streams. Data Min. Knowl. Discov. 30(3), 640–680 (2016)

    Article  MathSciNet  Google Scholar 

  3. Vatanen, T., Kuusela, M., Malmi, E., Raiko, T., Aaltonen, T., Nagai, Y.: Semi-supervised detection of collective anomalies with an application in high energy particle physics. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2012)

    Google Scholar 

  4. Gruhl, C., Sick, B., Tomforde, S.: Novelty detection in continuously changing environments. Futur. Gener. Comput. Syst. 114, 138–154 (2021)

    Article  Google Scholar 

  5. Gruhl, C., Sick, B.: Novelty detection with CANDIES: a holistic technique based on probabilistic models. Int. J. Mach. Learn. Cybern. 9(6), 927–945 (2018)

    Article  Google Scholar 

  6. Spinosa, E.J., de Carvalho, F., de Leon, A., Gama, J.: Novelty detection with application to data streams. Intell. Data Anal. 13(3), 405–422 (2009)

    Article  Google Scholar 

  7. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–58 (2009)

    Article  Google Scholar 

  8. Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 1–16 (2002)

    Google Scholar 

  9. Vakilinia, S., Zhang, X., Qiu, D.: Analysis and optimization of big-data stream processing. In: IEEE Global Communications Conference (GLOBECOM), pp. 1–6. IEEE (2016)

    Google Scholar 

  10. Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment. In: Proceedings of the International Conference on Very Large Data Bases (VLDB 2006), pp. 1–4 (2006)

    Google Scholar 

  11. Alzghoul, A., Löfstrand, M., Backe, B.: Data stream forecasting for system fault prediction. Comput. Ind. Eng. 62(4), 972–978 (2012)

    Article  Google Scholar 

  12. Gruhl, C.: Novelty detection in multivariate data stream with probabilistic models. Ph.D. thesis, University of Kassel (2022)

    Google Scholar 

  13. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  14. Zhao, Y., Nasrullah, Z., Li, Z.: PyOD: a Python toolbox for scalable outlier detection. J. Mach. Learn. Res. 20(96), 1–7 (2019). https://jmlr.org/papers/v20/19-011.html

  15. Lavin, A., Ahmad, S.: Evaluating real-time anomaly detection algorithms–the Numenta anomaly benchmark. In: ICMLA, pp. 38–44 (2016). arXiv:1510.03336

  16. Faria, E.R., Gama, J., Carvalho, A.C.: Novelty detection algorithm for data streams multi-class problems. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing, pp. 795–800 (2013)

    Google Scholar 

  17. Gruhl, C., Sick, B., Wacker, A., Tomforde, S., Hähner, J.: A building block for awareness in technical systems: online novelty detection and reaction with an application in intrusion detection. In: iCAST, pp. 194–200. IEEE (2015)

    Google Scholar 

  18. Fisch, D.: Intelligente technische Systeme mit der Fähigkeit zum kollaborativen Wissenserwerb, Dissertation, Universität Kassel (2011)

    Google Scholar 

  19. Hoffmann, H.: Kernel PCA for novelty detection. Pattern Recogn. 40(3), 863–874 (2007)

    Article  MATH  Google Scholar 

  20. Schölkopf, B., Williamson, R.C., Smola, A.J., Shawe-Taylor, J., Platt, J.C.: Support vector method for novelty detection. In: Advances in neural Information Processing Systems, vol. 12, pp. 582–588 (2000)

    Google Scholar 

  21. Gruhl, C., Tomforde, S.: Ohodin-online anomaly detection for data streams. In: 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C), pp. 193–197. IEEE (2021)

    Google Scholar 

  22. Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: SIGMOD, vol. 29, pp. 93–104. ACM (2000). https://dl.acm.org/citation.cfm?id=335191.335388

  23. Pevnỳ, T.: LODA: lightweight on-line detector of anomalies. Mach. Learn. 102(2), 275–304 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Liu, F.T., Ting, K.M., Zhou, Z.-H.: Isolation forest. In: Eighth IEEE International Conference on Data Mining, pp. 413–422. IEEE (2008)

    Google Scholar 

  25. Tan, S.C., Ting, K.M., Liu, T.F.: Fast anomaly detection for streaming data. In: Twenty-Second International Joint Conference on Artificial Intelligence, pp. 1511–1516 (2011)

    Google Scholar 

  26. Rousseeuw, P.J., Driessen, K.V.: A fast algorithm for the minimum covariance determinant estimator. Technometrics 41(3), 212–223 (1999)

    Article  Google Scholar 

  27. Quinn, J.A., Sugiyama, M.: A least-squares approach to anomaly detection in static and sequential data. Pattern Recogn. Lett. 40, 36–40 (2014)

    Article  Google Scholar 

  28. Ahmad, A., Bici, M., Campana, F.: Guidelines for topology optimization as concept design tool and their application for the mechanical design of the inner frame to support an ancient bronze statue. Appl. Sci. 11(17) (2021)

    Google Scholar 

  29. Verstraete, T., Coletti, F., Bulle, J., Vanderwielen, T., Arts, T.: Optimization of a U-bend for minimal pressure loss in internal cooling channels-Part I: Numerical method. J. Turbomach. 135(5) (2013)

    Google Scholar 

  30. Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics, Springer, Heidelberg (2006)

    MATH  Google Scholar 

  31. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)

    Google Scholar 

  32. Botache, D., et al.: Towards highly automated machine-learning-empowered monitoring of motor test stands. In: IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS), pp. 120–130 (2021)

    Google Scholar 

  33. Chicco, D., Jurman, G.: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21(1), 1–13 (2020)

    Article  Google Scholar 

Download references

Acknowledgment

This research has been partly funded by the German Ministry for Education and Research (BMBF) within the projects “Ein Organic-Computing-basierter Ansatz zur Sicherstellung und Verbesserung der Resilienz in technischen und IKT-Systemen (OCTIKT)” (01IS18064C) and the project “AI based Monitoring and Experimental Evaluation (AIMEE)” (01IS19061) and further funded by the Federal Ministry for Economic Affairs and Climate Action (BMWK) within the project “KI-basierte Topologieoptimierung elektrischer Maschinen (KITE)” (19I21034C).

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Kassel, Kassel, Germany

    Jens Decke, Jörn Schmeißing, Diego Botache, Bernhard Sick & Christian Gruhl

  2. Bosch Center for Artificial Intelligence, Hildesheim, Germany

    Maarten Bieshaar

Authors
  1. Jens Decke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jörn Schmeißing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diego Botache
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maarten Bieshaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bernhard Sick
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Christian Gruhl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jens Decke .

Editor information

Editors and Affiliations

  1. Technical University of Munich, Garching, Germany

    Martin Schulz

  2. Technical University of Munich, Heilbronn, Germany

    Carsten Trinitis

  3. Chalmers University of Technology, Gothenburg, Sweden

    Nikela Papadopoulou

  4. Otto-von-Guericke-Universität Magdeburg, Magdeburg, Germany

    Thilo Pionteck

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Decke, J., Schmeißing, J., Botache, D., Bieshaar, M., Sick, B., Gruhl, C. (2022). NDNET: A Unified Framework for Anomaly and Novelty Detection. In: Schulz, M., Trinitis, C., Papadopoulou, N., Pionteck, T. (eds) Architecture of Computing Systems. ARCS 2022. Lecture Notes in Computer Science, vol 13642. Springer, Cham. https://doi.org/10.1007/978-3-031-21867-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21867-5_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21866-8

  • Online ISBN: 978-3-031-21867-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation