Abstract
The last decade has witnessed tremendous growths of Internet of Things(IoT). Numerous condition monitoring systems(CMS) are widely applied to monitor equipments simultaneously. With the help of CMS, a large variety of monitor data from a large number of equipments can be collected in a very short time. However, it is a non-trivial task to take full advantage of such large amounts of monitor data in the context of anomaly detection. In this paper, we propose an approach called Latent Correlation based Anomaly Detection(LCAD) that can quickly detect potential anomalies from a large amount of monitor data, which posits that abnormal ones are a small portion in a mass of similar individuals. Instead of focusing on each single monitor data series, we identify the abnormal pattern by modeling the latent correlation among multiple correlative monitor data series using the Latent Correlation Probabilistic Model(LCPM), a probabilistic distribution model which can help to detect anomalies depending on their relations with LCPM. In order to validate our approach, we conduct experiments on the real-world datasets and the experimental results show that when facing a large amount of correlative monitor data series LCAD has a better performance as compared to the previous anomaly detection approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Atzori, L., Iera, A., Morabito, G.: The internet of things: A survey. Computer Networks 54(15), 2787–2805 (2010)
Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: Lof: identifying density-based local outliers. ACM Sigmod Record 29, 93–104 (2000)
Chan, P.K., Mahoney, M.V.: Modeling multiple time series for anomaly detection. In: Fifth IEEE International Conference on Data Mining, p. 8. IEEE (2005)
Dutton, W.H.: The internet of things. Available at SSRN (2013)
Eskin, E.: Anomaly detection over noisy data using learned probability distributions. In: Proceedings of the International Conference on Machine Learning (2000)
Eskin, E., Arnold, A., Prerau, M., Portnoy, L., Stolfo, S.: A geometric framework for unsupervised anomaly detection. In: Applications of Data Mining in Computer Security, pp. 77–101. Springer (2002)
González, F.A., Dasgupta, D.: Anomaly detection using real-valued negative selection. Genetic Programming and Evolvable Machines 4(4), 383–403 (2003)
Hawkins, S., He, H., Williams, G.J., Baxter, R.A.: Outlier detection using replicator neural networks. In: Kambayashi, Y., Winiwarter, W., Arikawa, M. (eds.) DaWaK 2002. LNCS, vol. 2454, pp. 170–180. Springer, Heidelberg (2002)
He, Z., Xu, X., Deng, S.: Discovering cluster-based local outliers. Pattern Recognition Letters 24(9), 1641–1650 (2003)
Otey, M.E., Ghoting, A., Parthasarathy, S.: Fast distributed outlier detection in mixed-attribute data sets. Data Mining and Knowledge Discovery 12(2-3), 203–228 (2006)
Papadimitriou, S., Sun, J., Faloutsos, C.: Streaming pattern discovery in multiple time-series. In: Proceedings of the 31st International Conference on Very Large Data Bases, pp. 697–708. VLDB Endowment (2005)
Ramaswamy, S., Rastogi, R., Shim, K.: Efficient algorithms for mining outliers from large data sets. ACM SIGMOD Record 29, 427–438 (2000)
Steinwart, I.: Consistency of support vector machines and other regularized kernel classifiers. IEEE Transactions on Information Theory 51(1), 128–142 (2005)
Tang, J., Chen, Z., Fu, A., Cheung, D.: A robust outlier detection scheme for large data sets. In: Proceedings of the 6th Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 535–548 (2002)
Yu, D., Sheikholeslami, G., Zhang, A.: Findout: finding outliers in very large datasets. Knowledge and Information Systems 4(4), 387–412 (2002)
Zhang, C., Weng, N., Chang, J., Zhou, A.: Detecting abnormal trend evolution over multiple data streams. In: Li, Q., Feng, L., Pei, J., Wang, S.X., Zhou, X., Zhu, Q.-M. (eds.) APWeb/WAIM 2009. LNCS, vol. 5446, pp. 285–296. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Ding, J., Liu, Y., Zhang, L., Wang, J. (2014). LCAD: A Correlation Based Abnormal Pattern Detection Approach for Large Amount of Monitor Data. In: Chen, L., Jia, Y., Sellis, T., Liu, G. (eds) Web Technologies and Applications. APWeb 2014. Lecture Notes in Computer Science, vol 8709. Springer, Cham. https://doi.org/10.1007/978-3-319-11116-2_51
Download citation
DOI: https://doi.org/10.1007/978-3-319-11116-2_51
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11115-5
Online ISBN: 978-3-319-11116-2
eBook Packages: Computer ScienceComputer Science (R0)