Abstract
We consider the stochastic optimal control problem of nonlinear mean-field systems in discrete time. We reformulate the problem into a deterministic control problem with marginal distribution as controlled state variable, and prove that dynamic programming principle holds in its general form. We apply our method for solving explicitly the mean-variance portfolio selection and the multivariate linear-quadratic McKean–Vlasov control problem.
Similar content being viewed by others
References
Ahmed, N.U., Ding, X.: Controlled McKean-Vlasov equation. Commun. Appl. Anal. 5, 183–206 (2001)
Andersson, D., Djehiche, B.: A maximum principle for SDEs of mean-field type. Appl. Math. Optim. 63, 341–356 (2010)
Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory. SpringerBriefs in Mathematics. Springer, New York (2013)
Bensoussan, A., Frehse, J., Yam, P.: The Master equation in mean-field theory. J. Math. Pures Appl. 103(6), 1441–1474 (2015)
Björk, T., Murgoci, A.: A general theory of Markovian time inconsistent stochastic control problems. Working Paper (2008)
Buckdahn, R., Djehiche, B., Li, J.: A general maximum principle for SDEs of mean-field type. Appl. Math. Optim. 64(2), 197–216 (2011)
Carmona, R., Delarue, F.: Forward-backward stochastic differential equations and controlled McKean Vlasov dynamics. Ann. Probab. 43(5), 2647–2700 (2015)
Carmona, R., Delarue, F., Lachapelle, A.: Control of McKean-Vlasov dynamics versus mean field games. Math. Financ. Econ. 7, 131–166 (2013)
Elliott, R., Li, X., Ni, Y.-H.: Discrete time mean-field stochastic linear-quadratic optimal control problems. Automatica 49, 3222–3233 (2013)
Huang, M., Caines, P., Malhamé, R.: Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inform. Syst. 6(3), 221–252 (2006)
Lasry, J.M., Lions, P.L.: Mean-field games. Jpn. J. Math. 2, 229–260 (2007)
Laurière, M., Pironneau, O.: Dynamic programming for mean-field type control. CRAS 352(9), 707–713 (2014)
Li, D., Zhou, X.Y.: Continuous-time mean-variance portfolio selection: a stochastic LQ framework. Appl. Math. Optim. 42, 19–33 (2000)
Pham, H., Wei, X.: Bellman equation and viscosity solutions for McKean-Vlasov control problem. arXiv:1512.07866, v2 (2015)
Sznitman, A.S.: Topics in Propagation of Chaos. Lecture Notes in Mathematics, vol. 1464, pp. 165–251. Springer, Heidelberg (1989)
Yong, J.: A linear-quadratic optimal control problem for mean-field stochastic differential equations. SIAM J. Control Optim. 51(4), 2809–2838 (2013)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pham, H., Wei, X. Discrete Time McKean–Vlasov Control Problem: A Dynamic Programming Approach. Appl Math Optim 74, 487–506 (2016). https://doi.org/10.1007/s00245-016-9386-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-016-9386-9