Abstract

In this paper a new approach to the design of S-model ergodic reinforcement learning algorithms is introduced. The new scheme utilizes a stochastic estimator and is able to operate in non-stationary environments with high accuracy and a high adaptation rate. According to the stochastic estimator scheme, which is the first attempt in the field, the estimates of the mean rewards of actions are computed stochastically. So, they are not strictly dependent on the environmental responses. The dependence between the stochastic estimates and the deterministic estimator's contents is more relaxed if the latter are not updated. In this way actions that have not been selected recently have the opportunity to be estimated as ‘optimal’, to increase their choice probability and consequently to be selected. Thus, the estimator is always recently updated and consequently able to adapt to environmental changes. The performance of the presented Stochastic Estimator Learning Automaton (SELA) is superior to all previous well-known S-model ergodic schemes. Furthermore it is proved that SELA is ε-optimal in every S-model random environment.

Author Keywords: Stochastic estimator; Learning window; Ergodic learning algorithm; Discretized learning algorithm; Probability slice

Article Outline

• References