Abstract
The paper deals with the numerical approximation of optimal strategies for two-person zero-sum differential games of pursuit-evasion type by neural networks. Thereby, the feedback strategies can be computed in real-time after the training of appropriate neural networks. For this purpose, sufficiently many optimal trajectories and their associated open-loop representations of the optimal feedback strategies must be computed, to provide data for training and cross-validation of the neural networks. All the precomputations can be carried through in a highly parallel way. This approach turns out to be applicable for differential games of more general type.
The method is demonstrated for a modified cornered rat game where a pursuing cat and an evading rat, both moving in simple motion, are constrained to a rectangular arena. Two holes in the walls surrounding the arena enable the rat to evade once and for all, if the rat is not too far from these holes. The optimal trajectories in the escape zone can be computed analytically. In the capture zone, a game of degree is employed with terminal time as payoff. To compute optimal trajectories for this secondary game, the time evolution of the survival region for the rat is determined via a sequence of discretized games.
The combination of these methods permits the computation of more than a thousand trajectories leading to some ten thousand sample patterns which relate the state variables to the values of the optimal strategies. These data exhibit characteristic properties of the optimal strategies. It is shown that these properties can be extracted from the data by use of neural networks. By means of the trained networks, about 200 trajectories arc finally simulated. The pursuer as well as the evader acts according to the controls proposed by the neural networks. Despite the simple structure of the neural networks used in this study, the strategics based upon them show a reasonable, close to optimal performance in a large variety of simulations of the pursuit-evasion game under consideration.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Anderson, B. D. O., Exponential Stability of Linear Equations Arising in Adaptive Identification, IEEE Transactions on Automatic Control 22, 1977, 83–88.
Ba§ar, T., and Olsder, G. J., Dynamic Noncooperative Game Theory, Academic Press, London, 1982, 2nd ed., 1994.
Benveniste, A., Metivier, M., and Priouret, P., Algorithmes adaptatifs et approximations stochastique, Masson, Paris, 1987.
Blum, E. K., and Li, L. K., Approximation Theory and Feedforward Networks, Neural Networks 4, 1990, 511 – 515.
Breakwell, J. V., Zero-Sum Differential Games with Terminal Payoff, in: P. Hagedorn, H. W. Knobloch, and G. J. Olsder (Eds.), Differential Games and Applications, Lecture Notes in Control and Information Sciences 3, Springer, Berlin, 1977, 70 – 95.
Breitner, M. H., Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances, in: R. Bulirsch, and D. Kraft (Eds.), Computational Optimal Control, International Series of Numerical Mathematics 115, Birkhauser, Basel, 1994, 147 – 162.
Breitner, M. H., Real-Time Capable Approximation of Optimal Strategies in Complex Differential Games, in: M. Breton, G. Zaccour (Eds.), Proceedings of the Sixth International Symposium on Dynamic Games and Applications, St-Jovite, Quebec, GERAD, Ecole des Hautes Etudes Commerciales, Montreal, 1994, 370 – 374.
Breitner, M. H., and Pesch H. J., Reentry Trajectory Optimization under Atmospheric Uncertainty as a Differential Game, in: T. Ba§ar, and A. Haurie (Eds.), Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games 1, Birkhauser, Boston, 1994, 70 – 88.
Breitner, M. H., Pesch, H. J., and Grimm, W., Complex Differential Games of Pursuit-Evasion Type with State Constraints, Part 1: Necessary Conditions for Optimal Open-Loop Strategies, Part 2: Numerical Computation of Optimal Open-Loop Strategies, Journal of Optimization Theory and Applications 78, 1993, 419–441, 443 – 463.
Bryson, A. E., and Ho, Y.-C., Applied Optimal Control, Hemisphere, New York, 1975.
Finnoff, W., Diffusion Approximations for the Constant Learning Rate Backpropagation Algorithm and Resistance to Local Minima, Neural Computation 6, 285 – 295, 1994.
Gabler, I., Numerische Berechnung optimaler Strategien eines Diffe-rentialspiels und Approximation durch Neuronale Netze, Diploma thesis, Department of Mathematics, Munich University of Technology, Munich, 1993.
Gabler, I., Miesbach, S., Breitner, M. H., and Pesch, H. J., Synthesis of Optimal Strategies for Differential Games by Neural Networks, Report No. 468, Deutsche Forschungsgemeinschaft, Schwer-punkt “Anwendungsbezogene Optimierung und Steuerung”, Department of Mathematics, Munich University of Technology, Munich, 1993.
Goh, C. J., and Edwards, N. J., Feedback Control of Minimum-Time Optimal Control Problems Using Neural Networks, Optimal Control Applications & Methods 14, 1993, 1 – 16.
Hertz, J., Krogh, A., and Palmer, R. G., Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, 1991.
Hornik, K., Stinchcombe, M., and White, H., Multi-Layer Feedforward Networks are Universal Approximators, Neural Networks 2, 1989, 359 – 366.
Isaacs, R., Differential Games, John Wiley & Sons, New York, 1965, Krieger, New York, 1975.
Jarmark, B., and Bengtsson, H., Near-Optimal Flight Trajectories Generated by Neural Networks, Internal Report, Saab/Scania Aircraft Division, Linkoping, 1989.
Koslik, B., Pesch, H. J., Breitner, M. H., and von Stryk, O., Optimal Control of a Complex Concern Model by Direct and Indirect Optimization Methods, to appear in: H. Engl, and H. Neunzert (Eds.), Proceedings of the 8th Conference of the European Consortium for Mathematics in Industry, ECMI Series, Teubner, Stuttgart, 1995.
Kugelmann, B., and Pesch, H. J., New General Guidance Method in Constrained Optimal Control, Part 1: Numerical Method, Part 2: Application to Space Shuttle Guidance, Journal of Optimization Theory and Applications 67, 1990, 421–435, 437 – 446.
Lachner, R., Breitner, M. H., and Pesch H. J., Three-Dimensional Air Combat: Numerical Solution of Complex Differential Games, in: G. J. Olsder (Ed.), New Trends in Dynamic Games and Applications, this volume, Birkhauser, Boston, 1995, 165 – 190.
Lachner, R., Breitner, M.H., and Pesch H.J., Optimal Strategies of a Complex Pursuit-Evasion Game, to appear in Journal of Computing and Information, 1995.
Miesbach, S., Bahnfuhrung von Robotern mit Neuronalen Netzen, PhD thesis, Department of Mathematics, Munich University of Technology, Munich, 1995.
Miesbach, S., and Schurmann, B., Wenn Roboter arbeiten lernen: Ideen und Methoden der Neuroinformatik zur Regelung und Steuerung, Informationstechnik 33, 1991, 300 – 309.
Miller, W. T., Sutton, R. S., and Werbos, P. J. (Eds.), Neural Networks for Control, MIT Press, Cambridge, 1990.
Nguyen, D., and Widrow, B., The Truck Backer-Upper: An Example of Self-Learning in Neural Networks, in: W. T. Miller, R. S. Sutton, and P. J. Werbos (Eds.), Neural Networks for Control, The MIT Press, Cambridge, 1990, 287 – 299.
Pesch, H. J., Real-Time Computation of Feedback Controls for Constrained Optimal Control Problems, Part 1: Neighboring Extremals, Part 2: A Correction Method Based on Multiple Shooting, Optimal Control Applications & Methods 10, 1989, 129–145, 147–171.
Pesch, H. J., Offline and Online Computation of Optimal Trajectories in the Aerospace Field, in: A. Miele and A. Salvetti (Eds.), Applied Mathematics in Aerospace Science and Engineering, Plenum, New York, 1994, 165–219.
Pesch, H. J., Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity, in: R. Bulirsch, and D. Kraft (Eds.), Computational Optimal Control, International Series of Numerical Mathematics 115, Birkhauser, Basel, 1994, 43–64.
Proceedings of the NIPS Neural Information Processing Systems, Morgan Kaufmann, San Mateo, CA, annually since 1987.
Ritter, H., Martinetz, T., and Schulten, K., Neuronale Netze: Eine Einfiihrung in die Neuroinformatik selbstorganisierender Netz-werke, Addison-Wesley, Bonn, 1991.
Rojas, R., Theorie der neuronalen Netze, Springer, Berlin, 1993.
Rosenblatt, F., Principles of Neurodynamics, Spartan, New York, 1962.
Saarinen, S., Bramley, R., and Cybenko, G., Ill-Conditioning in Neural Network Training Problems, SIAM Journal of Scientific and Computing 14, 693 – 714, 1993.
Stoer, J., and Bulirsch, R., Introduction to Numerical Analysis, Springer, New York, 2nd ed., 1993.
von Stryk, O., Numerical Solution of Optimal Control Problems by Direct Collocation, in: R. Bulirsch, A. Miele, J. Stoer, and K.-H. Well (Eds.), Optimal Control, International Series of Numerical Mathematics 111, Birkhauser, Basel, 1993, 129 – 143.
von Stryk, O., and Bulirsch, R., Direct and Indirect Methods for Trajectory Optimization, Annals of Operations Research 37, 1992, 357 – 373.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1995 Birkhäuser Boston
About this paper
Cite this paper
Pesch, H.J., Gabler, I., Miesbach, S., Breitner, M.H. (1995). Synthesis of Optimal Strategies for Differential Games by Neural Networks. In: Olsder, G.J. (eds) New Trends in Dynamic Games and Applications. Annals of the International Society of Dynamic Games, vol 3. Birkhäuser Boston. https://doi.org/10.1007/978-1-4612-4274-1_6
Download citation
DOI: https://doi.org/10.1007/978-1-4612-4274-1_6
Publisher Name: Birkhäuser Boston
Print ISBN: 978-1-4612-8719-3
Online ISBN: 978-1-4612-4274-1
eBook Packages: Springer Book Archive