Skip to main content
Log in

L2-Tracking of Gaussian Distributions via Model Predictive Control for the Fokker–Planck Equation

Vietnam Journal of Mathematics Aims and scope Submit manuscript

Abstract

This paper presents the first results for the stability analysis of Model Predictive Control schemes applied to the Fokker–Planck equation for tracking probability density functions. The analysis is carried out for linear dynamics and Gaussian distributions, where the distance to the desired reference is measured in the L2-norm. We present results for general such systems with and without control penalization. Refined results are given for the special case of the Ornstein–Uhlenbeck process. Some of the results establish stability for the shortest possible (discrete time) optimization horizon N = 2.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Notes

  1. The only exception are the results in our conference paper [10], which form a preliminary version of the results in this paper for a much more restricted class of systems.

  2. Note also that D has full rank.

  3. To see this in the equation for \(\dot {\Sigma }_{Y}(t),\) it is helpful to use (18), which holds due to Assumption 1(b).

  4. We remind that Σ(k) corresponds to the evaluation of Σ(t) at t = tk.

  5. In Fig. 2 (right), we have depicted the normalized differences (45) only for the first 10 steps as there are no visual changes afterwards.

References

  1. Altmüller, N., Grüne, L.: Distributed and boundary Model Predictive Control for the heat equation. GAMM-Mitt. 35, 131–145 (2012)

    Article  MathSciNet  Google Scholar 

  2. Annunziato, M., Borzì, A.: Optimal control of probability density functions of stochastic processes. Math. Model. Anal. 15, 393–407 (2010)

    Article  MathSciNet  Google Scholar 

  3. Annunziato, M., Borzì, A.: A Fokker–Planck control framework for multidimensional stochastic processes. J. Comput. Appl. Math. 237, 487–507 (2013)

    Article  MathSciNet  Google Scholar 

  4. Breiten, T., Kunisch, K., Pfeiffer, L.: Control strategies for the Fokker–Planck equation. ESAIM: Control, Optim Calc. Var. 24, 741–763 (2018)

    Article  MathSciNet  Google Scholar 

  5. Brockett, R.: Notes on the control of the Liouville equation. In: Alabau-Boussouira, F., et al. (eds.) Control of Partial Differential Equations, pp 101–129. Springer, Berlin (2012)

    Chapter  Google Scholar 

  6. Chen, Y., Georgiou, T.T., Pavon, M.: Optimal steering of a linear stochastic system to a final probability distribution, part II. IEEE Trans. Autom. Control 61, 1170–1180 (2016)

    Article  MathSciNet  Google Scholar 

  7. Dubljevic, S., Christofides, P.D.: Boundary predictive control of parabolic PDEs. In: 2006 American Control Conference, pp 49–56 (2006)

  8. Dubljevic, S., El-Farra, N.H., Mhaskar, P., Christofides, P.D.: Predictive control of parabolic PDEs with state and control constraints. Int. J. Robust Nonlinear Control 16, 749–772 (2006)

    Article  MathSciNet  Google Scholar 

  9. Feller, W.: Diffusion processes in one dimension. Trans. Am. Math. Soc. 77, 1–31 (1954)

    Article  MathSciNet  Google Scholar 

  10. Fleig, A., Grüne, L.: Estimates on the minimal stabilizing horizon length in Model Predictive Control for the Fokker–Planck equation. IFAC-PapersOnLine 49, 260–265 (2016)

    Article  MathSciNet  Google Scholar 

  11. Fleig, A., Grüne, L., Guglielmi, R.: Some results on Model Predictive Control for the Fokker–Planck equation. In: MTNS 2014: 21st International Symposium on Mathematical Theory of Networks and Systems, July 7–11, 2014, pp 1203–1206. University of Groningen, the Netherlands (2014)

  12. Fleig, A., Guglielmi, R.: Optimal control of the Fokker–Planck equation with space-dependent controls. J. Optim. Theory Appl. 174, 408–427 (2017)

    Article  MathSciNet  Google Scholar 

  13. Georgiou, T.T.: The structure of state covariances and its relation to the power spectrum of the input. IEEE Trans. Autom. Control 47, 1056–1066 (2002)

    Article  MathSciNet  Google Scholar 

  14. Gı̄hman, Ĭ.Ī., Skorohod, A.V.: Stochastic Differential Equations. Springer, New York (1972). Translated from the Russian by Kenneth Wickwire, Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 72

    Book  Google Scholar 

  15. Grüne, L.: Analysis and design of unconstrained nonlinear MPC schemes for finite and infinite dimensional systems. SIAM J. Control Optim. 48, 1206–1228 (2009)

    Article  MathSciNet  Google Scholar 

  16. Grüne, L., Pannek, J.: Nonlinear Model Predictive Control: Theory and Algorithms, 2nd edn. Springer, London (2017)

    Book  Google Scholar 

  17. Grüne, L., Pannek, J., Seehafer, M., Worthmann, K.: Analysis of unconstrained nonlinear MPC schemes with time varying control horizon. SIAM J. Control Optim. 48, 4938–4962 (2010)

    Article  MathSciNet  Google Scholar 

  18. Heemink, A.W.: Stochastic modelling of dispersion in shallow water. Stoch. Hydrol. Hydraul. 4, 161–174 (1990)

    Article  Google Scholar 

  19. Horsthemke, W., Lefever, R.: Noise-induced transitions: theory and applications in physics, Chemistry, and Biology. Springer Series in Synergetics, vol. 15. Springer, Berlin (1984)

    MATH  Google Scholar 

  20. Ito, K., Kunisch, K.: Receding horizon optimal control for infinite dimensional systems. ESAIM: Control Optim. Calc. Var. 8, 741–760 (2002)

    Article  MathSciNet  Google Scholar 

  21. Lincoln, B., Rantzer, A.: Relaxing dynamic programming. IEEE Trans. Autom. Control 51, 1249–1260 (2006)

    Article  MathSciNet  Google Scholar 

  22. Moreno, M.: A two-mean reverting-factor model of the term structure of interest rates. J. Futures Mark. 23, 1075–1105 (2003)

    Article  Google Scholar 

  23. Nkurunziza, S., Ahmed, S.E.: Shrinkage drift parameter estimation for multi-factor Ornstein–Uhlenbeck processes. Appl. Stoch. Models Bus. Ind. 26, 103–124 (2010)

    Article  MathSciNet  Google Scholar 

  24. Petersen, K.B., Pedersen, M.S.: The matrix cookbook. http://www2.imm.dtu.dk/pubdb/p.php?3274. Version 20121115 (2012)

  25. Primak, S., Kontorovich, V., Lyandres, V.: Stochastic Methods and Their Applications to Communications. Wiley, Hoboken (2004)

    Book  Google Scholar 

  26. Protter, P.E.: Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer, Berlin (2005)

    Book  Google Scholar 

  27. Rawlings, J., Mayne, D., Diehl, M.: Model Predictive Control: Theory and Design, 2nd edn. Nob Hill Publishing, Madison (2017)

    Google Scholar 

  28. Risken, H.: The Fokker–Planck equation. Springer Series in Synergetics, 2nd edn, vol. 18. Springer, Berlin (1989)

    Google Scholar 

  29. Roy, S., Annunziato, M., Borzì, A.: A Fokker–Planck feedback control-constrained approach for modelling crowd motion. J. Comput. Theor. Trans. 45, 442–458 (2016)

    Article  MathSciNet  Google Scholar 

  30. Tanaka, H.: Stochastic differential equations with reflecting boundary condition in convex regions. Hiroshima Math. J. 9, 163–177 (1979)

    MathSciNet  MATH  Google Scholar 

  31. Tröltzsch, F.: Optimal Control of Partial Differential Equations: Theory, Methods and Applications. Graduate Studies in Mathematics, vol. 112. American Mathematical Society, Providence, RI (2010)

    Google Scholar 

  32. Vasicek, O.: An equilibrium characterization of the term structure. J. Financ. Econ. 5, 177–188 (1977)

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Tobias Damm for very helpful suggestions and the referees for their valuable comments that helped to improve the manuscript.

Funding

This work was supported by the German Research Foundation (DFG) (grant number GR 1569/15-1).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Arthur Fleig.

Additional information

Dedicated to Prof. Hans Georg Bock on occasion of his 70th birthday.

Appendix

Appendix

Proof of Proposition 2

Due to Proposition 1, we can assume that μ̈ is arbitrarily close to \(\bar {\mu }= 0\). For |μ̈| sufficiently small, we argue below that the exponential controllability condition (24) with respect to stage cost (26) holds with C = 1 for the control candidate \((\bar {K},\bar {c})\). Then we apply Theorem 1 to conclude the assertion.

First, due to \(\bar {\mu }= 0\), we have that \(\bar {c}= 0\). Then, due to \(\bar {\Sigma }= 1\), we see from (13), (15), and (16) that applying \((\bar {K},\bar {c})\) results in

$$ \mu(t) = \mathring{\mu}e^{-2(\theta+\bar{K})t} \quad\text{and}\quad{\Sigma}(t) = 1+\left( \mathring{\Sigma} -1\right)e^{-2(\theta+\bar{K})t} > 0. $$

We define

$$\bar{\theta} := \theta+\bar{K}>0. $$

Then the stage cost (26) can be written as

$$\tilde{f}(t) \!:=\! 1\!+\left[\!1+\left( \!\mathring{\Sigma} -1\!\right)e^{-2\bar{\theta}t}\right]^{-\frac{1}{2}} - 2\!\left[\!\frac{2+\left( \!\mathring{\Sigma} -1\right)e^{-2\bar{\theta}t}}{2}\right]^{-\frac{1}{2}}\exp\!\left( \!-\frac{\mathring{\mu}^{2} e^{-2\bar{\theta}t}}{2(2+(\mathring{\Sigma}-1)e^{-2\bar{\theta}t})}\right) $$

(cf. Lemma 1.)

Our aim is to show \(\tilde {f}(t) \leq e^{-\kappa t}\tilde {f}(0)\) for some κ > 0 (for sufficiently small μ̈2). Then (24) holds with overshoot bound C = 1 and decay rate \(\delta =e^{-\kappa T_{s}}\). We claim that \(\tilde {f}(t) \leq e^{-\kappa t}\tilde {f}(0)\) with

$$\kappa:=\frac{\bar{\theta}}{\mathring{\Sigma}+ 1} > 0. $$

To this end, we prove \(\tilde {f}^{\prime }(t) + \kappa \tilde {f}(t) \leq 0\). First, to shorten the notation, we introduce

$$a := \mathring{\Sigma} -1\in(-1,\infty), \quad \tau := e^{-2\bar{\theta} t}\in(0,1], \quad \chi:=\frac{\mathring{\mu}^{2}\tau}{(a\tau+ 2)}\geq 0, $$
$$a_{1} := 2\sqrt{2}e^{-\chi/2}-\left( \frac{a\tau+ 2}{a\tau+ 1}\right)^{3/2}, \text{ and } a_{2} := 1-\frac{1}{(a\tau+ 1)^{3/2}}-\frac{4\sqrt{2}\chi e^{-\chi/2}(a + 2)}{(a\tau+ 2)^{3/2}}. $$

Then we can express \(\frac {\tilde {f}^{\prime }(t) + \kappa \tilde {f}(t)}{\bar {\theta }}\) by \(-\frac {h(\tau )}{{(a\tau + 2)}^{3/2}(a + 2)}\), where

$$h(\tau) := a_{1}\left( a\tau(a + 2)+a\tau+ 2\right)-a_{2}(a\tau+ 2)^{3/2}, $$

which means we have to show that h(τ) ≥ 0. We consider the two cases Σ̈ > 1 respective a > 0 and Σ̈ < 1 respective a < 0. The case Σ̈ = 1 is trivial.

First, let us assume a > 0. In this case, we set μ̈2 = εa for some ε ≥ 0. Then

$$\begin{array}{@{}rcl@{}} h(\tau) &=& a_{1}\left( a\tau(a + 2)+a\tau+ 2\right)-a_{2}(a\tau+ 2)^{3/2} \\ &\geq& a_{1}\left( a\tau(a + 2)+a\tau+ 2\right)-a_{3}(a\tau+ 2)^{3/2} \end{array} $$

with

$$a_{3} := 1-\frac{1}{(a\tau+ 1)^{3/2}}-\frac{4\sqrt{2}\chi e^{-\chi/2}}{(a\tau+ 2)^{1/2}} \geq a_{2} $$

due to a + 2 ≥ aτ + 2. If a1 ≥ 0, which we prove below, then

$$\begin{array}{@{}rcl@{}} h(\tau) &\geq& a_{1}(a\tau+ 2) + \underbrace{a_{1}a\tau(a + 2)}_{\geq a_{1}a\tau(a\tau+ 2)} - a_{3}(a\tau+ 2)^{3/2} \\ &\geq& \underbrace{(a\tau+ 2)}_{>0}\left( a_{1}+a_{1}a\tau-a_{3}\sqrt{a\tau+ 2}\right) \\ &=&(a\tau+ 2)\left( a_{1}(a\tau+ 1)-a_{3}\sqrt{a\tau+ 2}\right), \end{array} $$

i.e., it is left to show that \(a_{1}(a\tau + 1)-a_{2}\sqrt {a\tau + 2} \geq 0\). Furthermore, if a3 ≥ 0, then

$$\begin{array}{@{}rcl@{}} a_{1}(a\tau+ 1)-a_{3}\sqrt{a\tau+ 2} & \geq& a_{1}(a\tau+ 1)-a_{3}\left( \frac{a\tau}{2\sqrt{2}}+\sqrt{2}\right) \\ &=&a_{1}(a\tau+ 1)-\sqrt{2}a_{3}\left( \frac{a\tau}{4}+ 1\right) \\ &=&a_{1}(a\tau+ 1)-\sqrt{2}a_{3}\left( a\tau+ 1\right)+\frac{3}{4}\sqrt{2}a_{3}a\tau \\ &\geq& (a\tau+ 1)(a_{1}-\sqrt{2}a_{3}), \end{array} $$

reducing the problem further to

$$ a_{1}-\sqrt{2}a_{3} \geq 0. $$
(55)

Since a1 ≥ 0 follows from (55), we only need to prove (55) and a3 ≥ 0. Regarding the latter, with \(\tilde {a}:=a\tau \in [0,\infty )\) and for \(\varepsilon \in [0,\frac {1}{2}]\), we have

$$\begin{array}{@{}rcl@{}} a_{3} &=& 1-\frac{1}{(a\tau+ 1)^{3/2}}-\frac{4\sqrt{2}\chi e^{-\chi/2}}{(a\tau+ 2)^{1/2}} \\ &=& 1-\frac{1}{(\tilde{a}+ 1)^{3/2}}-\frac{4\sqrt{2}\varepsilon \tilde{a}}{(\tilde{a}+ 2)^{3/2}}\exp\left( -\frac{\varepsilon\tilde{a}}{2(\tilde{a}+ 2)}\right)\\ &\geq& 1-\frac{1}{(\tilde{a}+ 1)^{3/2}}-\frac{2\sqrt{2} \tilde{a}}{(\tilde{a}+ 2)^{3/2}}\exp\left( -\frac{\tilde{a}}{4(\tilde{a}+ 2)}\right) \geq 0, \end{array} $$

where the first inequality follows since a3 is monotonically decreasing in ε for \(\varepsilon \in [0,\frac {1}{2}]\):

$$\frac{\partial a_{3}}{\partial\varepsilon} = \underbrace{\frac{2\sqrt{2}\tilde{a}}{(\tilde{a}+ 2)^{5/2}}\exp\left( -\frac{\varepsilon\tilde{a}}{2(\tilde{a}+ 2)}\right)}_{\geq 0}\underbrace{\left[\varepsilon\tilde{a}-2(\tilde{a}+ 2)\right]}_{< 0} \leq 0. $$

Now, we can turn our attention to (55), which we claim holds for \(\varepsilon \in [0,\frac {1}{4}]\). With \(\tilde {a}=a\tau \) as above, we get

$$a_{1}-\sqrt{2}a_{3} = 2\sqrt{2}\exp\left( \!-\frac{\varepsilon \tilde{a}}{2(\tilde{a}+ 2)}\right)\!\left( \!1\!+\frac{2\sqrt{2}\varepsilon\tilde{a}}{(2+\tilde{a})^{3/2}}\right)-\left( \frac{\tilde{a}+ 2}{\tilde{a}+ 1}\right)^{\!3/2}-\sqrt{2}\left( \!1-\frac{1}{(\tilde{a}+ 1)^{3/2}}\right), $$

which unfortunately is not monotone with respect to ε. We know, however, that

$$ \left( a_{1}-\sqrt{2}a_{3}\right)_{|\tilde{a}= 0} = 0 \quad\text{and}\quad \left( a_{1}-\sqrt{2}a_{3}\right)\to\frac{2\sqrt{2}}{\sqrt{e^{\varepsilon}}}-(\sqrt{2}+ 1), \tilde{a}\to\infty, $$
(56)

where the limit is positive for \(\varepsilon \in [0,\frac {1}{4}]\). Moreover, in the special case ε = 0, we see that

$$ \frac{d (a_{1}-\sqrt{2}a_{3})}{d \tilde{a}} = \frac{3}{2(\tilde{a}+ 1)^{2}}\left( \sqrt{1+\frac{1}{\tilde{a}+ 1}}-\frac{\sqrt{2}}{\sqrt{\tilde{a}+ 1}}\right) \geq 0 ~\Leftrightarrow~ \frac{\tilde{a}}{\tilde{a}+ 1} \geq 0 ~\Leftrightarrow~ \tilde{a}\geq 0, $$
(57)

which, together with (56) proves that h(τ) ≥ 0 for ε = 0. In general, we have

$$ \frac{d (a_{1}-\sqrt{2}a_{2})}{d \tilde{a}}_{|\tilde{a}= 0} = \frac{3}{\sqrt{2}}\varepsilon. $$
(58)

A similar but more involved argument can be made to show that the derivative has at most one root for \(\tilde {a}>0\) and arbitrary but fixed \(\varepsilon \in [0,\frac {1}{4}]\). Then from (56) and (58) follows that h(τ) ≥ 0 for \(\varepsilon \in [0,\frac {1}{4}]\) and a > 0.

For a ∈ (− 1, 0), we cannot choose μ̈2 = εa. Instead, we set μ̈2 = ε ∈ [0, 1] and note that aτ ∈ (− 1, 0). Then

$$\begin{array}{@{}rcl@{}} h(\tau) &=& a_{1}\left( a\tau(a + 2)+a\tau+ 2\right)-a_{2}(a\tau+ 2)^{3/2} \\ &\geq& a_{1}\left( a\tau(a + 2)+a\tau+ 2\right)-a_{4}(a\tau+ 2)^{3/2} \end{array} $$

with

$$a_{4} := 1-\frac{1}{(a\tau+ 1)^{3/2}}-\frac{4\sqrt{2}\chi e^{-\chi/2}}{(a\tau+ 2)^{3/2}} $$

due to a < 1. If a1, a4 ≤ 0, then due to aτ ∈ (− 1, 0), we have

$$\begin{array}{@{}rcl@{}} &&a_{1}(a\tau+ 2) + \underbrace{a_{1}a\tau}_{\geq 0}\underbrace{(a + 2)}_{\geq 1} - a_{4}(a\tau+ 2)^{3/2} \geq a_{1}(a\tau+ 2) + a_{1}a\tau - a_{4}(a\tau+ 2)^{3/2} \\ =&& 2a_{1}(a\tau+ 1)\underbrace{-a_{4}}_{\geq 0}\underbrace{(a\tau+ 2)^{3/2}}_{\geq 2\sqrt{2}(a\tau+ 1)} \geq 2(a\tau+ 1)\left( a_{1}-\sqrt{2}a_{4}\right). \end{array} $$

Note that \((a\tau + 2)^{3/2} \geq 2\sqrt {2}(a\tau + 1)\) only holds for aτ ∈ (− 1, 0). We only show \(a_{1}-\sqrt {2}a_{4}\geq 0\) and a1 ≤ 0, since a4 ≤ 0 then follows. Regarding the latter, with μ̈2 = ε, we have

$$\begin{array}{@{}rcl@{}} a_{1} &=& 2\sqrt{2}e^{-\chi/2}-\left( \frac{a\tau+ 2}{a\tau+ 1}\right)^{\frac{3}{2}} = 2 \sqrt{2}\exp\left( -\frac{\varepsilon\tau}{2(a\tau+ 2)}\right)-\left( \frac{a\tau+ 2}{a\tau+ 1}\right)^{\frac{3}{2}}\\ &\leq& 2\sqrt{2}-\left( \frac{a\tau+ 2}{a\tau+ 1}\right)^{\frac{3}{2}} \leq 0. \end{array} $$

In the last step, we prove \(a_{1}-\sqrt {2}a_{4}\geq 0\):

$$\begin{array}{@{}rcl@{}} a_{1} - \sqrt{2}a_{4} &=& 2\sqrt{2}\exp\left( -\frac{\varepsilon \tau}{2(a\tau+ 2)}\right)\left( 1+\frac{2\sqrt{2}\varepsilon \tau}{(2+a\tau)^{\frac{5}{2}}}\right)\\&&-\left( \frac{a\tau+ 2}{a\tau+ 1}\right)^{\frac{3}{2}}-\sqrt{2}\left( 1-\frac{1}{(a\tau+ 1)^{\frac{3}{2}}}\right). \end{array} $$

One can set ε = −a ∈ (0, 1) and use \(\tilde {a}=a\tau \) to obtain a function depending only on one variable and prove the assertion directly. An alternative approach is to show that \(a_{1} - \sqrt {2}a_{4}\) is monotonously decreasing in ε for ε ∈ (0, 1), which is easy to show. Recall that this property did not hold in case of a > 1. Consequently, it suffices to consider ε = 0, for which

$$(a_{1}-\sqrt{2}a_{4})_{|\varepsilon= 0} = (a_{1}-\sqrt{2}a_{3})_{|\varepsilon= 0}. $$

In particular, we can use (57). Since the derivative is negative for \(\tilde {a}<0\) and the first equation in (56) holds, we have h(τ) ≥ 0 for a < 0. □

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fleig, A., Grüne, L. L2-Tracking of Gaussian Distributions via Model Predictive Control for the Fokker–Planck Equation. Vietnam J. Math. 46, 915–948 (2018). https://doi.org/10.1007/s10013-018-0309-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10013-018-0309-8

Keywords

Mathematics Subject Classification (2010)

Navigation