Optimal Control of Piecewise Deterministic Markov Processes

Abstract

This chapter studies the infinite-horizon continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. Two optimality criteria are considered, the discounted cost case and the long run average cost case. We provide conditions for the existence of a solution to an integro-differential optimality equality, the so called Hamilton-Jacobi-Bellman HJB) equation, for the discounted cost case, and a solution to an HJB inequality for the long run average cost case, aswell as conditions for the existence of a deterministic stationary optimal policy. From the results for the discounted cost case and under some continuity and compactness hypothesis on the parameters and non-explosive assumptions for the process, we derive the conditions for the long run average cost case by employing the so-called vanishing discount approach.