Abstract
There have been a great deal of researches based on deep neural networks for the network anomalies in the areas of network server workloads. We focus on deep neural networks to deal with huge server loads of network anomalies in a distributed MMOGs (massively multiplayer online games). We present an approach to training deep neural networks to control a loss function optimization, such as hyperparameters, using actor-critic methods from reinforcement learning (RL). Deep RL algorithms have been demonstrated on a range of major challenges: brittle convergence properties such as hyperparameter tuning. We propose an algorithm to automatically optimize the one of hyperparameters, Lagrangian multiplier of the support vector machines (SVM) deep neural networks using actor-critic algorithms. The setting of hyperparameters in the SVM deep neural networks is very important with regard to its accuracy and efficiency. Therefore, we employ the actor-critic algorithm to train Lagrangian multiplier of the SVM deep neural networks. We train a policy network called actor to decide the Lagrangian multiplier at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the Lagrangian multiplier given by the actor) that the actor made. Algorithm comparisons show that our algorithm leads to good optimization of Lagrangian multiplier and can prevent overfitting to a certain extent automatically without human system designers.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Silver D, Lever G, Heess N (2014) Deterministic policy gradient algorithms. In: ICML’14 proceedings of the 31st international conference on international conference on machine learning, vol 32, pp 387–395
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Driessche GVD, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489
Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. In: NIPS
Xu C, Qin T, Wang G, Liu T-Y (2017) Reinforcement learning for learning rate control, under review as a conference paper at ICLR 2017
Hansen S (2016) Using deep Q-Learning to control optimization hyperparameters, in ArXiv (2016)
Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machine. Mach Learn 46:389–422
Sutton RS (1988) Learning to predict by the methods of temporal differences. Mach Learn 3(1):9–44
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 1. MIT Press, Cambridge
Sutton RS, McAllester DA, Singh SP, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation, vol 99. In NIPS, pp 1057—1063
Sutton RS (1984) Temporal credit assignment in reinforcement learning. Doctoral Dissertation
Tieleman T, Hinton G (2012) Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2)
Tavallaee M, Bagheri E, Lu W, Ghorbani AA (2009) A detailed analysis of the KDD CUP 99 dataset. In: Proceedings of the 2009 IEEE symposium on computational intelligence in security and defense applications (CISDA 2009), pp 53–58
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2016RIA2B4012386).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kim, C., Park, Jm., Kim, Hy. (2019). An Actor-Critic Algorithm for SVM Hyperparameters. In: Kim, K., Baek, N. (eds) Information Science and Applications 2018. ICISA 2018. Lecture Notes in Electrical Engineering, vol 514. Springer, Singapore. https://doi.org/10.1007/978-981-13-1056-0_64
Download citation
DOI: https://doi.org/10.1007/978-981-13-1056-0_64
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1055-3
Online ISBN: 978-981-13-1056-0
eBook Packages: EngineeringEngineering (R0)