Abstract
We propose an on-line action-dependent heuristic dynamic programming approach based on recurrent neural network architecture – Echo state network (ESN) – as critic network within the frame of adaptive critic design (ACD), to be used for adaptive control. Here it is applied to the optimization of a complex nonlinear process for production of a biodegradable polymer, briefly called PHB. The on-line procedure for simultaneous critic training and process optimization is tested in the absence and presence of measurement noise. In both cases the optimization procedure succeeded in increasing the productivity and in proper training of the adaptive critic network at the same time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bellman, R.E.: Dynamic Programming. Princeton Univ. Press, Princeton (1957)
Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dymanic Programming. Athena Scientific, Belmont (1996)
Iyer, M.S., Wunsch II, D.C.: Dynamic reoptimization of a fed-batch fermentor using adaptive critic designs. IEEE Trans. on Neural Networks 12(6), 1433–1444 (2001)
Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the “echo state network” approach. GMD Report 159, German National Research Center for Information Technology (2002)
Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Advances in Neural Information Processing Systems (NIPS 2002), vol. 15, pp. 593–600. MIT Press, Cambridge (2003)
Koprinkova-Hristova, P.: ACD Approach to Optimal Control of Mixed Culture Cultivation for PHB Production Process – Sugar’s Time Profile Synthesis. In: Proc. of the IEEE Intelligent Systems IS 2008 Methodology, Models, Applications and Emerging Technologies, Varna, Bulgaria, September 6-8, vol. II, pp. 12-29–12-32 (2008)
Koprinkova-Hristova, P.: Knowledge-based approach to control of mixed culture cultivation for PHB production process. Biotechnology and Biotechnological Equipment 22(4), 964–967 (2008)
Koprinkova-Hristova, P., Oubbati, M., Palm, G.: Adaptive critic design with echo state network. In: 2010 IEEE Int. Conf. on Systems, Man and Cybernetics SMC 2010, Istanbul, Turkey, October 10-13 (2010) (accepted paper)
Lenardis, G.G.: A retrospective on adaptive dynamic programming for control. In: Proc. of Int. Joint Conf. on Neural Networks, Atlanta, GA, USA, June 14-19, pp. 1750–1757 (2009)
Lukosevicius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Computer Science Review 3, 127–149 (2009)
Patnaik, P.R.: Neural network designs for poly-β-hydroxybutyrate production optimization under simulated industrial conditions. Biotechnology Letters 27, 409–415 (2005)
Patnaik, P.R.: Genetic algorithmic optimization of PHB production by a mixed culture in an optimally dispersed fed-batch bioreactor. Int. J. Boiautomation 13(3), 27–46 (2009)
Prokhorov, D.V.: Adaptive critic designs and their applications. Ph.D. dissertation. Department of Electrical Engineering, Texas Tech. Univ. (1997)
Prokhorov, D.: Echo state networks: appeal and challenges. In: Proc. of Int. Joint Conf. on Neural Networks (IJCNN), Montreal, Canada, pp. 1463–1466 (2005)
Prokhorov, D.: Toward effective combination of off-line and on-line training in ADP framework. In: Proc. of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL 2007), pp. 268–271 (2007)
Prokhorov, D.: Training recurrent neurocontrollers for real-time applications. IEEE Trans. on Neural Networks 18(4), 1003–1015 (2007)
Schrauwen, B., Wandermann, M., Verstraeten, D., Steil, J.J.: Improving reservoirs using intrinsic plasticity. Neurocomputing 71, 1159–1171 (2008)
Si, J., Wang, Y.-T.: On-line learning control by associationand reinforcement. IEEE Trans. on Neural Networks 12(2), 264–276 (2001)
Simple and very simple Matlab toolbox for Echo State Networks by H. Jaeger and group members, http://www.reservoir-computing.org/software
Sutton, R.S.: Learning to predict by methods of temporal differences. Machine Learning 3, 9–44 (1988)
Tohyama, M., Patarinska, T., Qiang, Z., Shimizu, K.: Modeling of the mixed culture and periodic control for PHB production. Biochemical Engineering Journal 10, 157–173 (2002)
Werbos, P.J.: Backpropagation through time: What it does and how to do it. Proceedings of the IEEE 78(10), 1550–1560 (1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Koprinkova-Hristova, P., Palm, G. (2010). Adaptive Critic Design with ESN Critic for Bioprocess Optimization. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds) Artificial Neural Networks – ICANN 2010. ICANN 2010. Lecture Notes in Computer Science, vol 6353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15822-3_54
Download citation
DOI: https://doi.org/10.1007/978-3-642-15822-3_54
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15821-6
Online ISBN: 978-3-642-15822-3
eBook Packages: Computer ScienceComputer Science (R0)