Abstract

In a Markov decision problem with hidden state variables, a posterior distribution serves as a state variable and Bayes’ law under an approximating model gives its law of motion. A decision maker expresses fear that his model is misspecified by surrounding it with a set of alternatives that are nearby when measured by their expected log likelihood ratios (entropies). Martingales represent alternative models. A decision maker constructs a sequence of robust decision rules by pretending that a sequence of minimizing players choose increments to martingales and distortions to the prior over the hidden state. A risk sensitivity operator induces robustness to perturbations of the approximating model conditioned on the hidden state. Another risk sensitivity operator induces robustness to the prior distribution over the hidden state. We use these operators to extend the approach of Hansen and Sargent [Discounted linear exponential quadratic Gaussian control, IEEE Trans. Automat. Control 40(5) (1995) 968–971] to problems that contain hidden states.

Keywords: Robustness; Hidden state Markov chains; Martingales; Risk sensitivity; Decision theory

JEL classification codes: C32; C52; D81; D83; D84