Abstract
Generative adversarial network (GAN) is the most exciting machine learning breakthrough in recent years, and it trains the learning model by finding the Nash equilibrium of a two-player zero-sum game. GAN is composed of a generator and a discriminator, both trained with the adversarial learning mechanism. In this paper, we introduce and investigate the use of GAN for novelty detection. In training, GAN learns from ordinary data. Then, using previously unknown data, the generator and the discriminator with the designed decision boundaries can both be used to separate novel patterns from ordinary patterns. The proposed GAN-based novelty detection method demonstrates a competitive performance on the MNIST digit database and the Tennessee Eastman (TE) benchmark process compared with the PCA-based novelty detection methods using Hotelling’s T2 and squared prediction error statistics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadi M, Andersen D, 2016. Learning to protect communications with adversarial neural cryptography. https://arxiv.org/abs/1610.06918
Arjovsky M, Chintala S, Bottou L, 2017. Wasserstein generative adversarial networks. Int Conf on Machine Learning, p.214–223.
Berthelot D, Schumm T, Metz L, 2017. BEGAN: boundary equilibrium generative adversarial networks. https://arxiv.org/abs/1703.10717
Clifton L, Clifton D, Watkinson P, et al., 2011. Identification of patient deterioration in vital-sign data using one-class support vector machines. Federated Conf on Computer Science and Information Systems, p.125–131.
Denton E, Chintala S, Fergus R, et al., 2015. Deep generative image models using a Laplacian pyramid of adversarial networks. Advances in Neural Information Processing Systems, p.1486–1494.
Donahue J, Krähenbühl P, Darrell T, 2016. Adversarial feature learning. https://arxiv.org/abs/1605.09782
Downs J, Vogel E, 1993. A plant-wide industrial process control problem. Comput Chem Eng, 17(3):245–255. https://doi.org/10.1016/0098-1354(93)80018-I
Dumoulin V, Belghazi I, Poole B, et al., 2016. Adversarially learned inference. https://arxiv.org/abs/1606.00704
Ge Z, Song Z, 2013. Bagging support vector data description model for batch process monitoring. J Proc Contr, 23(8):1090–1096. https://doi.org/10.1016/j.jprocont.2013.06.010
Ge Z, Yang C, Song Z, 2009. Improved kernel PCA-based monitoring approach for nonlinear processes. Chem Eng Sci, 64(9):2245–2255. https://doi.org/10.1016/j.ces.2009.01.050
Ge Z, Gao F, Song Z, 2011. Batch process monitoring based on support vector data description method. J Proc Contr, 21(6):949–959. https://doi.org/10.1016/j.jprocont.2011.02.004
Ge Z, Song Z, Gao F, 2013. Review of recent research on data-based process monitoring. Ind Eng Chem Res, 52(10):3543–3562. https://doi.org/10.1021/ie302069q
Ge Z, Demyanov S, Chen Z, et al., 2017. Generative Open-Max for multi-class open set classification. https://arxiv.org/abs/1707.07418
Goodfellow I, Pouget-Abadie J, Mirza M, et al., 2014. Generative adversarial nets. Advances in Neural Information Processing Systems, p.2672–2680.
Grover A, Ermon S, 2017. Boosted generative models. https://arxiv.org/abs/1702.08484
Hautamaki V, Karkkainen I, Franti P, 2004. Outlier detection using k-nearest neighbour graph. Proc 17th Int Conf on Pattern Recognition, p.430–433. https://doi.org/10.1109/ICPR.2004.1334558
He Z, Deng S, Xu X, 2005. An optimization model for outlier detection in categorical data. LNCS, 3644:400–409. https://doi.org/10.1007/11538059_42
Hoffmann H, 2007. Kernel PCA for novelty detection. Patt Recogn, 40(3):863–874. https://doi.org/10.1016/j.patcog.2006.07.009
Kadurin A, Aliper A, Kazennov A, et al., 2017a. The cornucopia of meaningful leads: applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget, 8(7):10883. https://doi.org/10.18632/oncotarget.14073
Kadurin A, Nikolenko S, Khrabrov K, et al., 2017b. dru-GAN: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico. Mol Pharmaceut, 14(9):3098–3104. https://doi.org/10.1021/acs.molpharmaceut.7b00346
Keogh E, Lonardi S, Ratanamahatana C, 2004. Towards parameter-free data mining. Proc 10th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.206–215. https://doi.org/10.1145/1014052.1014077
Kim T, Cha M, Kim H, et al., 2017. Learning to discover cross-domain relations with generative adversarial networks. https://arxiv.org/abs/1703.05192
Ledig C, Theis L, Huszár F, et al., 2016. Photo-realistic single image super-resolution using a generative adversarial network. https://arxiv.org/abs/1609.04802
Li J, Liang X, Wei Y, et al., 2017. Perceptual generative adversarial networks for small object detection. CVPR, p.1951–1959. https://doi.org/10.1109/CVPR.2017.211
Li Y, Maguire L, 2011. Selecting critical patterns based on local geometrical and statistical information. IEEE Trans Patt Anal Mach Intell, 33(6):1189–1201. https://doi.org/10.1109/TPAMI.2010.188
Li Y, Liu S, Yang J, et al., 2017. Generative face completion. CVPR, p.5892–5900. https://doi.org/10.1109/CVPR.2017.624
Luc P, Couprie C, Chintala S, et al., 2016. Semantic segmentation using adversarial networks. https://arxiv.org/abs/1611.08408
Mahadevan S, Shah S, 2009. Fault detection and diagnosis in process data using one-class support vector machines. J Proc Contr, 19(10):1627–1639. https://doi.org/10.1016/j.jprocont.2009.07.011
Mao X, Li Q, Xie H, et al., 2016. Least squares generative adversarial networks. https://arxiv.org/abs/1611.04076
Mogren O, 2016. C-RNN-GAN: continuous recurrent neural networks with adversarial training. https://arxiv.org/abs/1611.09904
Patcha A, Park J, 2007. An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput Netw, 51(12):3448–3470. https://doi.org/10.1016/j.comnet.2007.02.001
Pimentel M, Clifton D, Clifton L, et al., 2014. A review of novelty detection. Signal Process, 99:215–249. https://doi.org/10.1016/j.sigpro.2013.12.026
Radford A, Metz L, Chintala S, 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. https://arxiv.org/abs/1511.06434
Reed S, Akata Z, Yan X, et al., 2016. Generative adversarial text to image synthesis. Proc 33rd Int Conf on Machine Learning, p.1060–1069.
Schlegl T, Seeböck P, Waldstein S, et al., 2017. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. Int Conf on Information Processing in Medical Imaging, p.146–157. https://doi.org/10.1007/978-3-319-59050-9_12
Springenberg J, 2015. Unsupervised and semi-supervised learning with categorical generative adversarial networks. https://arxiv.org/abs/1511.06390
Vondrick C, Pirsiavash H, Torralba A, 2016. Generating videos with scene dynamics. Advances in Neural Information Processing Systems, p.613–621.
Wu J, Zhang C, Xue T, et al., 2016. Learning a probabilistic latent space of object shapes via 3D generativeadversarial modeling. Advances in Neural Information Processing Systems, p.82–90.
Xiao Y, Wang H, Xu W, et al., 2016. Robust one-class SVM for fault detection. Chemometr Intell Lab Syst, 151: 15–25. https://doi.org/10.1016/j.chemolab.2015.11.010
Yang Z, Chen W, Wang F, et al., 2017. Improving neural machine translation with conditional sequence generative adversarial nets. https://arxiv.org/abs/1703.04887
Yeh R, Chen C, Lim T, et al., 2016. Semantic image inpainting with perceptual and contextual losses. https://arxiv.org/abs/1607.07539
Yi Z, Zhang H, Gong P, et al., 2017. DualGAN: unsupervised dual learning for image-to-image translation. https://arxiv.org/abs/1704.02510
Yu J, 2012. Semiconductor manufacturing process monitoring using Gaussian mixture model and Bayesian method with local and nonlocal information. IEEE Trans Semicond Manuf, 25(3):480–493. https://doi.org/10.1109/TSM.2012.2192945
Yu J, Qin S, 2008. Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models. AIChE J, 54(7):1811–1829. https://doi.org/10.1002/aic.11515
Yu J, Qin S, 2009. Multiway Gaussian mixture model based multiphase batch process monitoring. Ind Eng Chem Res, 48(18):8585–8594. https://doi.org/10.1021/ie900479g
Yu L, Zhang W, Wang J, et al., 2017. SeqGAN: sequence generative adversarial nets with policy gradient. 31st AAAI Conf on Artificial Intelligence, p.2852–2858.
Zhao F, Feng J, Zhao J, et al., 2018. Robust LSTMautoencoders for face de-occlusion in the wild. IEEE Trans Image Process, 27(2):778–790. https://doi.org/10.1109/TIP.2017.2771408
Zhao J, Mathieu M, LeCun Y, 2016. Energy-based generative adversarial network. https://arxiv.org/abs/1609.03126
Zhu J, Park T, Isola P, et al., 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. https://arxiv.org/abs/1703.10593
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, Hg., Li, X. & Zhang, T. Generative adversarial network based novelty detection usingminimized reconstruction error. Frontiers Inf Technol Electronic Eng 19, 116–125 (2018). https://doi.org/10.1631/FITEE.1700786
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.1700786