Abstract
In this paper, we propose a new unsupervised anomaly detection framework for detecting network intrusions online. The framework consists of new anomalousness metrics named IP Weight and an outlier detection algorithm based on Gaussian mixture model (GMM). IP Weights convert the features of IP packets into a four-dimensional numerical feature space, in which the outlier detection takes place. Intrusion decisions are made based on the outcome of outlier detections. Two sets of experiments are conducted to evaluate our framework. In the first experiment, we conduct an offline evaluation based on the 1998 DARPA intrusion detection dataset, which detects 16 types of attacks out of a total of 19 network attack types. In the second experiment, an online evaluation is performed in a live networking environment. The evaluation result not only confirms the detection effectiveness with DARPA dataset, but also shows a strong runtime efficiency, with response times falling within seconds.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson, J.P.: Computer Security Threat Monitoring and Surveillance. Technical Report, James P. Anderson Co., Fort Washington, Pennsylvania (1980)
Axelsson, S.: The Base-Rate Fallacy and the Difficulty of Intrusion Detection. ACM Transactions on Information and System Security (TISSEC) 3, 186–201 (2000)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm (with discussion). Journal of the Royal Statistical Society B 39, 1–38 (1977)
Denning, D.E.: An Intrusion Detection Model. IEEE Transactions on Software Engineering 2, 222–232 (1987)
Eskin, E.: Anomaly Detection over Noisy Data using Learned Probability Distributions. In: Proceedings of 17th International Conference on Machine Learning, pp. 255–262 (2000)
Eskin, E., Arnold, A., Prerau, M., Portnoy, L., Stolfo, S.: A Geometric Framework for Unsupervised Anomaly Detection: Detecting Intrusions in Unlabeled Data. In: On Application of Data Mining in Computer Security. Kluwer Academic Publisher, Dordrecht (2002)
Fluxay, http://www.netxeyes.com
Frank, J.: Artificial Intelligence and Intrusion Detection: Current and Future Directions. In: Proceedings of the 17th National Computer Security Conference, pp. 11–21 (1994)
Forrest, S., Hofmeyr, S.A., Longstaff, T.A.: A Sense of Self for Unix Processes. In: Proceedings of 1996 IEEE Symposium on Security and Privacy, pp. 120–128 (1996)
Hochberg, J., Jackson, K., Stallings, C., McClary, J.F., DuBois, D., Ford, J.: NADIR: An Automated System for Detecting Network Intrusion and Misuse. Computers & Security 12(3), 235–248 (1993)
Kendall, K.: A Database of Computer Attacks for the Evaluation of Intrusion Detection Systems. Master’s Thesis, Massachusetts Institute of Technology (1998)
Lunt, T., Jagannathan, R., Lee, R., Listgarten, S., Edwards, D., Neumann, P., Javitz, H., Valdes, A.: IDES: The Enhanced Prototype, A Real-time Intrusion Detection System. Technical Report, SRI Project 4185-010, Computer Science Laboratory, CA (1988)
McHugh, J.: The 1998 Lincoln Lab IDS Evaluation - A Critique. In: Debar, H., Mé, L., Wu, S.F. (eds.) RAID 2000. LNCS, vol. 1907, pp. 145–161. Springer, Heidelberg (2000)
Portnoy, L., Eskin, E., Stolfo, S.: Intrusion Detection with Unlabeled Data using Clustering. In: Proceedings of ACM CSS Workshop on Data Mining Applied to Security (2001)
Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)
Roberts, F.S.: Measurement Theory. Addison-Wesley Publishing Company, Reading (1979)
Smaha, S.E.: Haystack: An Intrusion Detection System. In: Proceedings of the IEEE Fourth Aerospace Computer Security Applications Conference, pp. 37–44 (1988)
Titterington, D., Smith, A., Makov, U.: Statistical Analysis of Finite Mixture Distributions. John Wiley & Sons, New York (1985)
X-scan, http://www.xfocus.org
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lu, W., Traore, I. (2005). A New Unsupervised Anomaly Detection Framework for Detecting Network Attacks in Real-Time. In: Desmedt, Y.G., Wang, H., Mu, Y., Li, Y. (eds) Cryptology and Network Security. CANS 2005. Lecture Notes in Computer Science, vol 3810. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11599371_9
Download citation
DOI: https://doi.org/10.1007/11599371_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-30849-2
Online ISBN: 978-3-540-32298-6
eBook Packages: Computer ScienceComputer Science (R0)