Abstract
In this paper we study two-person nonzero-sum games for continuous-time jump processes with the randomized history-dependent strategies under the finite-horizon payoff criterion. The state space is countable, and the transition rates and payoff functions are allowed to be unbounded from above and from below. Under the suitable conditions, we introduce a new topology for the set of all randomized Markov multi-strategies and establish its compactness and metrizability. Then by constructing the approximating sequences of the transition rates and payoff functions, we show that the optimal value function for each player is a unique solution to the corresponding optimality equation and obtain the existence of a randomized Markov Nash equilibrium. Furthermore, we illustrate the applications of our main results with a controlled birth and death system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aliprantis, C., Border, K.: Infinite Dimensional Analysis. Springer, New York (2007)
Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer, Berlin (2011)
Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific, Belmont (1996)
Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley, Hoboken (2005)
Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, New York (1997)
Guo, X.P., Hernández-Lerma, O.: Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs. J. Appl. Probab. 42, 303–320 (2005)
Guo, X.P., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes: Theory and Applications. Springer, Berlin (2009)
Guo, X.P., Huang, Y.H., Song, X.Y.: Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent policies. SIAM J. Control Optim. 50, 23–47 (2012)
Guo, X.P., Huang, X.X., Huang, Y.H.: Finite horizon optimality for continuous-time Markov decision processes with unbounded transition rates. Adv. Appl. Probab. (2015, to appear)
Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999)
Kitaev, M.Y.: Semi-Markov and jump Markov controlled models: average cost criterion. Theory Probab. Appl. 30, 272–288 (1985)
Kitaev, M.Y., Rykov, V.V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)
Royden, H.L.: Real Analysis. Macmillan, New York (1988)
Shapley, L.: Stochastic games. Proc. Natl. Acad. Sci. 39, 1095–1100 (1953)
Wei, Q.D., Chen, X.: Continuous-time Markov decision processes with finite-horizon expected total cost criteria (2014). arXiv:1408.5497
Zhang, W.Z., Guo, X.P.: Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates. Sci. China Math. 55, 2405–2416 (2012)
Acknowledgments
The research was supported by China Postdoctoral Science Foundation (No. 2015M570007). We are greatly indebted to the associate editor and the anonymous referees for the valuable comments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wei, Q., Chen, X. Stochastic Games for Continuous-Time Jump Processes Under Finite-Horizon Payoff Criterion. Appl Math Optim 74, 273–301 (2016). https://doi.org/10.1007/s00245-015-9314-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-015-9314-4
Keywords
- Nonzero-sum games
- Finite-horizon payoff criterion
- Unbounded transition rates
- Randomized history-dependent strategies
- Nash equilibrium