Constrained Adaptive Testing with Shadow Tests

Abstract

The intuitive principle underlying adaptive testing is that a test has better measurement properties if the difficulties of its items match the ability of the examinee. Items that are too easy or difficult have predictable responses and cannot provide much information about the ability of the examinee. The first to formalize this principle was Birnbaum (1968). The information measure he used was Fisher’s well-known information in the sample. For dichotomous response models, the measure is defined as

$$I(\theta) = \sum\limits^n_{i=1} I_i(\theta)= \sum\limits^n_{i=1}\frac{(P^\prime(\theta))^2}{P(\theta)[1 - P(\theta)]},$$

(2.1)

where P _i (θ) is the probability of a correct response to item i = 1, …, n for an examinee with ability θ, I _i (θ) is the information in the examinee’s response to item i, and I(θ) is the information in his or her joint responses to the test.