Abstract
Supervised Classification algorithms are only trained to recognize and classify certain patterns, those contained in the training group. Therefore, these will by default, classify the unknown patterns incorrectly, causing unwanted results. This work proposes several solutions, to make the referred algorithms capable of detecting unknown patterns. The main approach for the development of models capable of recognizing these patterns, was the use of three different models of Autoencoders: Simple Autoencoder (SAE), Convolutional Autoencoder (CAE) and Variational Autoencoder (VAE), that are a specific type of Neural Networks. After carrying out several tests on each of the three models of Autoencoders, it was possible to determine which one performed best the task of detecting/rejecting atypical patterns. Afterwards, the performance of the best Autoencoder was compared to the performance of a Convolutional Neural Network (CNN) in the execution of the referred task. The conclusion was that the VAE effectively detected atypical patterns better than the CNN. Some conventional Machine Learning techniques (Support Vector Machine (SVM), Random Forest (RF), Logistic Regression (LR)) were also tested. The one that presented the best performance was the RF classifier, achieving an accuracy of 75% in the detection of atypical/typical patterns. Thus, regarding the classification balance between atypical and typical patterns, Machine Learning techniques were not enough to surpass the Deep Learning methods, where the best accuracy reached 88% for the VAE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Holzinger, A., Kieseberg, P., Weippl, E., Tjoa, A.M.: Current advances, trends and challenges of machine learning and knowledge extraction: from machine learning to explainable AI. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2018. LNCS, vol. 11015, pp. 1–8. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99740-7_1
Abiodun, O.I., Jantan, A., Omolara, A.E., Dada, K.V., Mohamed, N.A., Arshad, H.: State-of-the-art in artificial neural network applications: a survey. Heliyon 4(11), e00938 (2018)
Rezaei, M., Shahidi, M.: Zero-shot learning and its applications from autonomous vehicles to COVID-19 diagnosis: a review. Intell.-Based Med. 3–4, 100005 (2020). https://doi.org/10.1016%2Fj.ibmed.2020.100005
Lampert, C.H., Nickisch, H., Harmeling, S.: Attribute-based classification for zero-shot visual object categorization. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 453–465 (2014)
Larochelle, H., Erhan, D., Bengio, Y.: Zero-data learning of new tasks. In: Proceedings of the 23rd National Conference on Artificial Intelligence - Volume 2, ser. AAAI 2008, pp. 646–651. AAAI Press (2008)
Rohrbach, M., Stark, M., Schiele, B.: Evaluating knowledge transfer and zero-shot learning in a large-scale setting. In: CVPR 2011, pp. 1641–1648 (2011)
Yu, X., Aloimonos, Y.: Attribute-based transfer learning for object categorization with zero/one training example. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 127–140. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15555-0_10
Cordella, L., De Stefano, C., Tortorella, F., Vento, M.: A method for improving classification reliability of multilayer perceptrons. IEEE Trans. Neural Networks 6(5), 1140–1147 (1995)
De Stefano, C., Sansone, C., Vento, M.: To reject or not to reject: that is the question-an answer in case of neural classifiers. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 30(1), 84–94 (2000)
Geifman, Y., El-Yaniv, R.: Selective classification for deep neural networks (2017). https://arxiv.org/abs/1705.08500
Baldi, P.: Autoencoders, unsupervised learning, and deep architectures. In: Guyon, I., Dror, G., Lemaire, V., Taylor, G., Silver, D. (eds.) Proceedings of ICML Workshop on Unsupervised and Transfer Learning, ser. Proceedings of Machine Learning Research, vol. 27, Bellevue, Washington, USA: PMLR, 02 Jul 2012, pp. 37–49. https://proceedings.mlr.press/v27/baldi12a.html
An, J., Cho, S.: Variational autoencoder based anomaly detection using reconstruction probability. Spec. Lect. IE 2(1), 1–18 (2015)
Bhandare, A., Bhide, M., Gokhale, P., Chandavarkar, R.: Applications of convolutional neural networks. Int. J. Comput. Sci. Inf. Technol. 7(5), 2206–2215 (2016)
Cheng, W., Hüllermeier, E.: Combining instance-based learning and logistic regression for multilabel classification. Mach. Learn. 76(2), 211–225 (2009)
Wijaya, A., Bisri, A.: Hybrid decision tree and logistic regression classifier for email spam detection. In: 2016 8th International Conference on Information Technology and Electrical Engineering (ICITEE), pp. 1–4 (2016)
Sahin, Y., Duman, E.: Detecting credit card fraud by ann and logistic regression. In: International Symposium on Innovations in Intelligent Systems and Applications 2011, pp. 315–319 (2011)
H.-L. Hwa, et al.: Prediction of breast cancer and lymph node metastatic status with tumour markers using logistic regression models. J. Eval. Clin. Pract. 14(2), 275–280 (2008). https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-2753.2007.00849.x
Bisong, E.: Logistic regression. In: Bisong, E. (ed.) Building Machine Learning and Deep Learning Models on Google Cloud Platform, pp. 243–250. Apress, Berkeley (2019). https://doi.org/10.1007/978-1-4842-4470-8_20
Cervantes, J., Garcia-Lamont, F., Rodríguez-Mazahua, L., Lopez, A.: A comprehensive survey on support vector machine classification: applications, challenges and trends. Neurocomputing 408, 189–215 (2020). https://www.sciencedirect.com/science/article/pii/S0925231220307153
Wang, Z., Xue, X.: Multi-class support vector machine. In: Ma, Y., Guo, G. (eds.) Support Vector Machines Applications, pp. 23–48. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-02300-7_2
Vaibhaw, Sarraf, J., Pattnaik, P.: Brain-computer interfaces and their applications. In: Balas, V.E., Solanki, V.K., Kumar, R. (eds.) An Industrial IoT Approach for Pharmaceutical Industry Growth, pp. 31–54. Academic Press (2020). https://www.sciencedirect.com/science/article/pii/B9780128213261000024
Belgiu, M., Drăguţ, L.: Random forest in remote sensing: a review of applications and future directions. ISPRS J. Photogramm. Remote. Sens. 114, 24–31 (2016)
Caie, P.D., Dimitriou, N., Arandjelović, O.: Artificial intelligence and deep learning in pathology. Elsevier 114, 149–173 (2021)
Freund, Y., Schapire, R.E.: A short introduction to boosting. J. Jpn. Soc. Artif. Intell. 14, 771–780 (1999)
Kaya, A., Keceli, A.S., Catal, C., Tekinerdogan, B.: Model analytics for defect prediction based on design-level metrics and sampling techniques. In: Model Management and Analytics for Large Scale Systems, pp. 125–139 (2020)
LeCun, Y.: The MNIST database of handwritten digits (1998). http://yann.lecun.com/exdb/mnist/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lopes, J.C., Rodrigues, P.J.S. (2022). Techniques to Reject Atypical Patterns. In: Pereira, A.I., Košir, A., Fernandes, F.P., Pacheco, M.F., Teixeira, J.P., Lopes, R.P. (eds) Optimization, Learning Algorithms and Applications. OL2A 2022. Communications in Computer and Information Science, vol 1754. Springer, Cham. https://doi.org/10.1007/978-3-031-23236-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-23236-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23235-0
Online ISBN: 978-3-031-23236-7
eBook Packages: Computer ScienceComputer Science (R0)