Abstract
Recently, both supervised and unsupervised deep learning techniques have accomplished notable results in various fields. However neural networks with back-propagation are liable to trapping at local minima. Genetic algorithms have been popular as a class of optimization techniques which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum.
In this paper, a variable length chromosome genetic algorithm assisted deep autoencoder is proposed. Firstly, the training of autoencoder is done with the help of variable length chromosome genetic algorithm. Secondly, a classifier is used for the classification of encoded data and compare the classification accuracy with other state-of-the-art methods. The experimental results show that the proposed method achieves competitive results and produce sparser networks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bengio, Y., LeCun, Y., et al.: Scaling learning algorithms towards AI. Large-scale Kernel Mach. 34, 1–41 (2007)
Utgoff Hinton, G.E., Osindero, S., Teh, Y.-W.: Many-layered learning. Neural Comput. 14, 2497–2529 (2002). MIT Press
Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006). MIT Press
Freund, Y., Haussler, D.: Unsupervised learning of distributions of binary vectors using two layer networks, Computer Research Laboratory, University of California, Santa Cruz (1994)
Bengio, Y., Lamblin, P., Dan, P., et al.: Greedy layer-wise training of deep networks. In: Advances in Neural Information Processing Systems, vol. 19, p. 153. MIT Press (2007)
Ranzato, M., Poultney, C., Chopra, S., et al.: Efficient learning of sparse representations with an energy-based model. In: Proceedings of NIPS (2007)
Bengio, Y., et al.: Learning deep architectures for AI. In: Foundations and Trends in Machine Learning, vol. 2, pp. 1–127. Now Publishers, Inc. (2009)
Weston, J., Ratle, F., Mobahi, H., Collobert, R.: Deep learning via semi-supervised embedding. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 639–655. Springer, Heidelberg (2012). doi:10.1007/978-3-642-35289-8_34
Mobahi, H., Collobert, R., Weston, J.: Deep learning from temporal coherence in video. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 737–744. ACM (2009)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks, vol. 313, pp. 504–507. American Association for the Advancement of Science (2006)
Vincent, P., Larochelle, H., Bengio, Y., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)
Ahmed, A., Yu, K., Xu, W., Gong, Y., Xing, E.: Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 69–82. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88690-7_6
Osindero, S., Hinton, G.E.: Modeling image patches with a directed hierarchy of Markov random fields. In: Advances in Neural Information Processing Systems, pp. 1121–1128 (2008)
Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning, pp. 791–798. ACM (2007)
Hinton, G.E., Salakhutdinov, R.R.: Using deep belief nets to learn covariance kernels for Gaussian processes. In: Advances in Neural Information Processing Systems, pp. 1249–1256 (2008)
Levner, I.: Data Driven Object Segmentation. Citeseer (2009)
Mnih, A., Hinton, G.E.: A scalable hierarchical distributed language model. In: Advances in Neural Information Processing Systems, pp. 1081–1088 (2009)
Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)
Ranzato, M.A., Szummer, M.: Semi-supervised learning of compact document representations with deep networks. In: Proceedings of the 25th International Conference on Machine Learning, pp. 792–799. ACM (2008)
David, O.E., Greental, I.: Genetic algorithms for evolving deep neural networks. In: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1451–1452. ACM (2014)
LeCun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998). IEEE
David, S.J., Whitley, D., Eshelman, L.J.: Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Combinations of Genetic Algorithms and Neural Networks, pp. 1–37. IEEE (1992)
Golberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Boston (1989)
Koehn, P.: Combining Genetic Algorithms and Neural Networks: The Encoding Problem. Citeseer (1994)
Schiffmann, W., Joost, M., Werner, R.: Application of genetic algorithms to the construction of topologies for multilayer perceptrons. In: Albrecht, R.F., Reeves, C.R., Steele, N.C. (eds.) Artificial Neural Nets and Genetic Algorithms, pp. 675–682. Springer, Wien (1993). doi:10.1007/978-3-7091-7533-0_98
Hancock, P.J.B., Smith, L.S.: Gannet: genetic design of a neural net for face recognition. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 292–296. Springer, Heidelberg (1991). doi:10.1007/BFb0029766
Bishop, J.M., Bushnell, M.J., Usher, A., et al.: Genetic optimisation of neural network architectures for colour recipe prediction. In: Albrecht, R.F., Reeves, C.R., Steele, N.C. (eds.) Artificial Neural Nets and Genetic Algorithms, pp. 719–725. Springer, Wien (1993). doi:10.1007/978-3-7091-7533-0_104
Montana, D.J., Davis, L.: Training feedforward neural networks using genetic algorithms. In: IJCAI 1989, vol. 89, pp. 762–767 (1989)
Zhang, M., Deng, Y., Chang, D.: A novel genetic clustering algorithm with variable-length chromosome representation. In: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1483–1484. ACM (2014)
Yahya, A.A., Osman, A., Ramli, A.R., et al.: Feature selection for high dimensional data: an evolutionary filter approach. Citeseer (2011)
Salakhutdinov, R., Hinton, G.: Semantic hashing. Int. J. Approximate Reasoning 50, 969–978 (2009). Elsevier
Brie, A.H., Morignot, P.: Genetic planning using variable length chromosomes. In: ICAPS 2005, pp. 320–329 (2005)
Baldi, P.: Autoencoders, unsupervised learning, and deep architectures. In: ICML Unsupervised and Transfer Learning, vol. 27, p. 1 (2012)
Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 61271374).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Sami Ullah Khan, Q., Li, J., Zhao, S. (2017). Training Deep Autoencoder via VLC-Genetic Algorithm. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10635. Springer, Cham. https://doi.org/10.1007/978-3-319-70096-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-70096-0_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70095-3
Online ISBN: 978-3-319-70096-0
eBook Packages: Computer ScienceComputer Science (R0)