Elsevier

Information Sciences

Volume 31, Issue 3, December 1983, Pages 243-263
Information Sciences

On conditional optimality of a class of learning automata in random environments

https://doi.org/10.1016/0020-0255(83)90055-5Get rights and content

Abstract

In this paper, we develop a new class of learning automata to operate in a random environment with a finite number of responses, called a Q-model, and investigate its asymptotic behavior from the viewpoint of conditional optimality. It is shown that this class contains as a particular case the β-model automaton introduced by Luce, which is available for an environment with binary responses, called a P-model. Making use of the martingale convergence theorem for the asymptotic analysis of the automata, we show that the automata are conditionally optimal, i.e., the automata converge optimally under the restrictions on the probability distributions that characterize the environment. The usefulness of the result is illustrated by its application to two simple automata: the β-model automaton and another one which is also applied to a P-model environment. Then, extending the idea underlying the second automaton, we develop a design method for an effective and structurally simple automaton for any Q-model environment. The conditional optimality of the resulting automaton is also investigated.

References (9)

  • K.S. Narendra et al.

    Learning automata—a survey

    IEEE Trans. Systems Man Cybernet.

    (1974)
  • I.J. Shapiro et al.

    Use of stochastic automata for parameter self-optimization

    IEEE Trans. Syst. Sci. Cybernet.

    (1969)
  • R. Viswanathan et al.

    A note on the linear reinforcement scheme for variable structure stochastic automata

    IEEE Trans. Systems Man Cybernet.

    (1972)
  • S. Lakshmivarahan

    Learning Algorithms Theory and Applications

    (1981)
There are more references available in the full text version of this article.

Cited by (0)

View full text