Abstract
We consider a risk-sensitive continuous-time Markov decision process over a finite time duration. Under the conditions that can be satisfied by unbounded transition and cost rates, we show the existence of an optimal policy, and the existence and uniqueness of the solution to the optimality equation out of a class of possibly unbounded functions, to which the Feynman–Kac formula was also justified to hold.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bäuerle N, Rieder U (2014) More risk-sensitive Markov decision processes. Math Oper Res 39:105–120
Bäuerle N, Popp A (2018) Risk-sensitive stopping problems for continuous-time Markov chains. Stochastics 90:411–431
Cavazos-Cadena R, Montes-de-Oca R (2000) Optimal stationary policies in risk-sensitive dynamic programs with finite state space and nonnegative rewards. Appl Math 27:167–185
Cavazos-Cadena R, Montes-de-Oca R (2000) Nearly optimal policies in risk-sensitive positive dynamic programming on discrete spaces. Math Methods Oper Res 52:133–167
Ghosh M, Saha S (2014) Risk-sensitive control of continuous time Markov chains. Stochastics 86:655–675
Guo X, Zhang Y (2018) On risk-sensitive piecewise deterministic Markov decision processes. Appl. Math Optim. in press. https://doi.org/10.1007/s00245-018-9485-x
Guo XP, Huang X, Huang Y (2015) Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates. Adv Appl Probab 47:1064–1087
Guo XP, Piunovskiy A (2011) Discounted continuous-time Markov decision processes with constraints: unbounded transition and loss rates. Math Oper Res 36:105–132
Hernández-Lerma O, Lasserre J (1996) Discrete-time Markov control processes. Springer, New York
Hernández-Lerma O, Lasserre J (1999) Further topics on discrete-time Markov control processes. Springer, New York
Howard R, Matheson J (1972) Risk-sensitive Markov decision proceses. Manag Sci 18:356–369
Jacod J (1975) Multivariate point processes: predictable projection, Radon–Nicodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie und verwandte Gebiete 31:235–253
Jaśkiewicz A (2008) A note on negative dynamic programming for risk-sensitive control. Oper Res Lett 36:531–534
Kitaev M (1986) Semi-Markov and jump Markov controlled models: average cost criterion. Theory Probab Appl 30:272–288
Kitaev M, Rykov V (1995) Controlled queueing systems. CRC Press, New York
Kumar KS, Chandan P (2013) Risk-sensitive control of jump process on denumerable state space with near monotone cost. Appl Math Optim 68:311–331
Patek S (2001) On terminating Markov decision processes with a risk-averse objective function. Automatica 37:1379–1386
Piunovski A, Khametov V (1985) New effective solutions of optimality equations for the controlled Markov chains with continuous parameter (the unbounded price-function). Problems Control Inform Theory 14:303–318
Piunovskiy A, Zhang Y (2011) Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach. SIAM J Control Optim 49:2032–2061
Piunovskiy A, Zhang Y (2014) Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR-Q J Oper Res 12, 4975
Wei Q (2016) Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion. Math Methods Oper Res 84:461–487
Wei Q, Chen X (2016) Continuous-time Markov decision processes under the risk-sensitive average cost criterion. Oper Res Lett 44:457–462
Zhang Y (2017) Continuous-time Markov decision processes with exponential utility. SIAM J Control Optim 55:2636–2660
Acknowledgements
This work is partially supported by Natural Science Foundation of Guangdong Province (Grant No. 2014A030313438), Zhujiang New Star (Grant No. 201506010056), Guangdong Province outstanding young teacher training plan (Grant No. YQ2015050).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
There is no potential conflicts of interest.
Ethical standard
Research do not have human participants and/or animals.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Guo, X., Liu, Q. & Zhang, Y. Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates. 4OR-Q J Oper Res 17, 427–442 (2019). https://doi.org/10.1007/s10288-019-0398-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10288-019-0398-6