Abstract
Current monitoring of IP flow records is challenged by the required analysis of large volume of flow records. Finding essential information is equivalent to searching for a needle in a haystack. This analysis can reach from simple counting of basic flow level statistics to complex data mining techniques. Some key target objectives are for instance the identification of malicious traffic as well as tracking the cause of observed flow related events. This paper investigates the usage of link analysis based methods for ranking IP flow records. We leverage the well known HITS algorithm in the context of flow level dependency graphs. We assume a simple dependency model that can be build in the context of large scale IP flow record data. We apply our approach on several datasets, ranging from ISP captured flow records up to forensic packet captures from a real world intrusion.
Chapter PDF
References
Aguilera, M.K., Mogul, J.C., Wiener, J.L., Reynolds, P., Muthitacharoen, A.: Performance debugging for distributed systems of black boxes. In: Proceedings of the nineteenth ACM symposium on Operating systems principles, pp. 74–89 (2003)
Barham, P., Black, R., Goldszmidt, M., Isaacs, R., MacCormick, J., Mortier, R., Simma, A.: Constellation: automated discovery of service and host dependencies in networked systems. TechReport, MSR-TR-2008-67 (2008)
Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems 30(1-7), 107–117 (1998)
Chen, M., Accardi, A., Kiciman, E., Lloyd, J.: Path-based failure and evolution management. In: NSDI 2004 (January 2004)
Chen, X., Zhang, M., Mao, Z.M., Bahl, P.: Automating network application dependency discovery: Experiences, limitations, and new solutions. In: Proceedings of OSDI (2008)
Internet Engineering Task Force(IETF). Ip flow information export (ipfix) (March 2010), http://www.ietf.org/dyn/wg/charter/ipfix-charter.html
Network Working Group. Rpc: Remote procedure call protocol specification version 2 (March 2010), http://tools.ietf.org/html/rfc5531
Iliofotou, M., Pappu, P., Faloutsos, M., Mitzenmacher, M., Singh, S., Varghese, G.: Network monitoring using traffic dispersion graphs (tdgs). In: Proceedings of the 7th ACM SIGCOMM conference on Internet measurement, pp. 315–320 (2007)
Jian-Guang, L., Qiang, F., Yi Wang, J.: Mining dependency in distributed systems through unstructured logs analysis, http://research.microsoft.com
Kandula, S., Chandra, R., Katabi, D.: What’s going on?: learning communication rules in edge networks. In: Proceedings of the ACM SIGCOMM 2008 conference on Data communication, pp. 87–98 (2008)
Kannan, J., Jung, J., Paxson, V., Koksal, C.E.: Semi-automated discovery of application session structure. In: Proceedings of the 6th ACM SIGCOMM conference on Internet measurement, pp. 119–132 (2006)
Kleinberg, J.: Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM) 46(5) (September 1999)
Ng, A.Y., Zheng, A.X., Jordan, M.I.: Link analysis, eigenvectors and stability. In: International Joint Conference on Artificial Intelligence, vol. 17(1), pp. 903–910 (2001)
The Honeynet Project. Scan18 (March 2010), http://old.honeynet.org/scans/scan18/
Reynolds, P., Wiener, J.L., Mogul, J.C., Aguilera, M.K., Vahdat, A.: Wap5: black-box performance debugging for wide-area systems. In: Proceedings of the 15th international conference on World Wide Web, pp. 347–356 (2006)
Sawilla, R., Ou, X.: Identifying critical attack assets in dependency attack gaphs. In: Jajodia, S., Lopez, J. (eds.) ESORICS 2008. LNCS, vol. 5283, pp. 18–34. Springer, Heidelberg (2008)
Wang, S., State, R., Ourdane, M., Engel, T.: Mining netflow records for critical network activities. In: Proceedings of the 6th International Wireless Communications & Mobile Computing Conference (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 IFIP International Federation for Information Processing
About this paper
Cite this paper
Wang, S., State, R., Ourdane, M., Engel, T. (2010). Mining NetFlow Records for Critical Network Activities. In: Stiller, B., De Turck, F. (eds) Mechanisms for Autonomous Management of Networks and Services. AIMS 2010. Lecture Notes in Computer Science, vol 6155. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13986-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-13986-4_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13985-7
Online ISBN: 978-3-642-13986-4
eBook Packages: Computer ScienceComputer Science (R0)