Abstract

Most problems in learning to control dynamic systems involve learning under uncertainty, noise, and the lack of explicit instructional information about how to perform a task. Under these circumstances, techniques developed by artificial intelligence researchers for ‘learning from examples’, including the ‘supervised learning’ techniques studied by neural network researchers, are impractical because of the difficulty of obtaining training information (the ‘examples’) in the form of situation-action training pairs. A useful alternative in such situations is a learning technique that can discover appropriate actions in various situations through a search process that is guided by evaluative performance feedback. Reinforcement learning methods developed by neural network researchers are examples of such techniques. This paper focuses on direct reinforcement learning techniques and discusses their role in learning control by relating them to similar adaptive control methods. Several examples are also presented to illustrate the power and utility of direct reinforcement learning techniques for learning control.

Author Keywords: Reinforcement learning; Learning control; Direct methods; Peg-in-hole insertion; Inverse kinematics; Pole balancing

Article Outline

• References