Distribution of judgements in adaptive testing

Abstract

In a psychophysical experiment the observer is given the task to detect a signal or to discriminate two signals; the performance of the observer in this task is to be determined as a function of the physical parameters of the stimuli presented. To increase efficiency of testing, one stimulus parameter is controlled in such a way that the task is made more difficult when the observer performs well and made easier when he doesn't. The observer then forms part of a feedback loop, a part that is characterized by stochastic behaviour. In a well-designed experiment the parameter under study will continually oscillate around its “target value” (i.e., the threshold of detectability or of discriminability). This paper derives an expression for the equilibrium distribution of this parameter around the target point, valid for an extremely long experiment. The observer is described as a stochastic detector with stationary characteristics. In the experiment he is subjected to a number of trials with stimuli having constant parameters. A set of rules applied to the performance of the subject in a group of trials then determines in which direction the parameter under study is to be varied for the next group of trials. For a wide class of experimental designs the desired equilibrium distribution of this parameter can be found from a recurrence formula. The derivation of this formula is given.

Examples of application of this theory are presented. Equilibrium distributions computed from the recurrence formula are found to be good predictors of results of experiments carried out with a finite length. This result allows a deeper study of what constitutes the best procedural design in a given case. It is finally shown how the class of usable procedures can be enlarged by judicious combination of different types of rules.