Abstract
The paradigm of Markov jump process is widely employed in disparate contexts, like for instance in ecology, epidemiology and chemistry, any time the state-space is discrete and a tracked entity (the system) stochastically jumps from site to site. The classical master equation describes the system’s evolution in terms of site occupation probabilities starting from a given initial condition associated with the initial knowledge about the system. For the master equation of Markov processes admitting stationary occupation probabilities, it is here derived an equivalent quadratic format in an extended space of mutually interrelated variables having physical dimension of inverse of time. The evolution in the probability space is thus mirrored by the evolution in the extended space. This universal format potentially allows one to unveil general traits underlying the master equation dynamics. Here we specifically consider the emergence of an intrinsic rate which, behaving as a state function in the probability space, introduces a timing during the relaxation process. This specific feature has to be taken an empirical discovery which derives from the analysis of numerical calculations; a possible direction towards a formal proof is however proposed. The conjecture made here is that such intrinsic timing is a typical trait (i.e., normally present) of the Markov jump processes.
Similar content being viewed by others
Notes
As later discussed in Sect. 2.1, multiple stationary states can be admitted in our analysis. This means that the values \(p_i^\textrm{ss}\) can generally depend on the initial condition.
In the specific context of chemical kinetics, other ways of quadratization have been devised by exploiting smooth transformations of the variables; see for instance Ref. [39].
From Eq. 14 we have that \(z_{(j, \alpha \beta )} = p_j(t)^{-1} dp_j(t)/dt - p_\alpha (t)^{-1} dp_\alpha (t)/dt\). The critical cases are those for which j and/or \(\alpha \) correspond to sites depopulated as \(t \rightarrow \infty \), so that both the occupation probability and its rate of variation tend to zero. Let s be a site of such a kind. We can see that \(p_s(t)^{-1} dp_s(t)/dt\) tends to a finite limit value. Indeed, \(p_s(t)\) must monotonically decay to zero after a certain time \(t^*\), hence \(dp_s(t)/dt < 0\) for \(t \ge t^*\). Then, indicating with \(k^\textrm{exit}_s = \sum _{i \ne s} k_{s \rightarrow i}\) the total depopulation rate of the site s, from Eq. 1 we have that \(dp_s(t)/dt = -k^\textrm{exit}_s p_s(t) + \sum _{i \ne s} p_i(t) k_{i \rightarrow s} > -k^\textrm{exit}_s p_s(t)\). From the two inequalities taken together we obtain \(-k^\textrm{exit}_s< p_s(t)^{-1} dp_s(t)/dt < 0\) for any \(t > t^*\). It follows that also the limit value for \(t \rightarrow \infty \) is bounded. This implies that also the “critical” \(z_{(j, \alpha \beta )}\) have finite values \(z_{(j, \alpha \beta )}^*\).
By noting that \(|V_{(j, \alpha \beta ), (j', \alpha ' \beta ')} |\) is either equal to 0 or to \(h_{(j', \alpha '\beta ')} = p_{\alpha '} k_{\alpha ' \rightarrow \beta '} / p_{j'}\), it follows that \(\sum _{\alpha ', \beta '} |V_{(j, \alpha \beta ), (j', \alpha ' \beta ')}|= c/ p_{j'}\) with the constant factor \(c = \sum _{\alpha ', \beta '} p_{\alpha '} k_{\alpha ' \rightarrow \beta '}\). This factor can be eliminated by exploiting the normalization \(\sum _{j'} p_{j'} = 1\). Ultimately, this leads to \(p_{j'} = \gamma _{j'} / \sum _{j''} \gamma _{j''}\) where \(\gamma _{j'} = \left( \sum _{\alpha ',\beta '} |V_{(j, \alpha \beta ), (j', \alpha ' \beta ')}|\right) ^{-1} \;\;\; \mathrm{for \; any} \; j, \alpha , \beta \). The condition “for any \(j, \alpha , \beta \)” is automatically fulfilled because of the mutual interrelations among the elements of the matrix \(\textbf{V}\). In fact, while the forward transformation \(\textbf{p} \rightarrow \textbf{V}\) specifies unequivocally the matrix \(\textbf{V}\) once the site-site connections are given, the backward transformation is meaningful only if the elements of \(\textbf{V}\) are properly interrelated.
In the ‘hyper-spherical representation’, the system’s state is specified by the variables \((S, {\varvec{\psi }})\) where \({\varvec{\psi }}\) is a state vector normalized as \({\varvec{\psi }}^T {\varvec{\psi }}= 1\) and \(S = \sqrt{\textrm{Tr}\{ \textbf{V}^T \textbf{V}\}} >0\) is the Frobenius norm of the matrix \(\textbf{V}\). The \({\varvec{\psi }}\) components are interpreted as angular-like variables, and S as a radial-like variable. The evolution law has a simple and appealing format which resembles similar laws encountered in the modeling of the evolution of living species in fitness landscapes [54, 55]; see remarks in the Supplementary Material. Of note, the hyper-spherical representation enabled to unveil the existence of orthogonal attracting subspaces in the hyper-spherical space [32, 35].
Computations have been done with double-precision digits. For a cross-check of the accuracy, the time propagation was done also with the implicit ‘Variable Coefficient ODE Solver’ (VODE) [56] for stiff kinetics. The double-precision Fortran77 subroutine ‘DVODE.f’ was employed. The routine is freely available at the URL: https://computing.llnl.gov/projects/odepack/software (last viewed on 28th November 2022). The results were coincident under the graphical resolution of the plots.
Given two arrays \(\textbf{a}\) and \(\textbf{b}\) of equal dimension, the dyadic product is here meant to be the direct product \(\textbf{a} \otimes \textbf{b}\) arranged as square matrix with elements \(a_i b_j\).
Let us consider the scaled quantities \(\alpha _Q^{(n)}(\textbf{p}) := z_Q^{(n)}(\textbf{p})/\max _{Q} \{ |z_Q^{(n)}(\textbf{p}) |\}\). It was found that, for most of the \(\textbf{p}\), the \(\alpha _Q^{(n)}(\textbf{p})\) either go to zero or to some constant value as n increases. In particular, in most cases it was found that \(\alpha _Q^{(n)}(\textbf{p}) \pm 1\). An automatic analysis was conducted for \(n=50\) to inspect on the typicality of this behaviour. The situation \(\alpha _Q^{(n)}(\textbf{p}) \pm 1\) was considered to be occurred if all \(|\alpha _Q^{(n)}(\textbf{p}) |\) were below 0.1 or above 0.9. For \(N = 3\), with jump rates generated at random over a range of 10 orders of magnitude with uniform sampling on the logarithmic scale, and also generating \(\textbf{p}\) at random, the percentage of instances in which such behavior was not observed was of about \(0.7\%\). The percentage increased to about \(1.4 \%\) for \(N = 4\), and to \(2.1 \%\) for \(N = 5\) and \(N = 6\). The number of generated instances was \(10^4\) for \(N = 3, 4\) and \(10^3\) for \(N = 5, 6\). In a relevant fraction of the cases, however, there were indications that \(n = 50\) was presumably too low, and that the expected behavior would have been likely observed going to higher orders.
The following criterion was adopted to assess the convergence to \(\tilde{\textbf{W}}_\infty \). For the elements \((\tilde{\textbf{W}}_{n-1})_{ij} \ne 0\), let us consider the relative variation \(\epsilon _{ij} = |(\tilde{\textbf{W}}_n)_{ij} - (\tilde{\textbf{W}}_{n-1})_{ij} |/(\tilde{\textbf{W}}_{n-1})_{ij}\). The convergence of \(\tilde{\textbf{W}}_n\) to \(\tilde{\textbf{W}}_\infty \) was considered to be achieved (or much likely about to be achieved) if \(\max _{i,j} \{ \epsilon _{ij} \} \le 0.05\) for some \(n \le n_\textrm{max}\) where \(n_\textrm{max}\) is a chosen cutoff. The analysis was done with \(n_\textrm{max} = 10\). Note that this criterion is quite stringent since the elements of \(\tilde{\textbf{W}}_{n-1}\) that possibly go slowly to zero should be neglected to avoid an apparent convergence failure. Moreover, in many cases the convergence could have been likely achieved going to orders much above \(n_\textrm{max}\). Indeed, in the inspected cases the fraction of instances in which the check passed with \(n = n_\textrm{max}\) was of about \(15-20 \%\), which suggests that many instances of missed convergence were merely due to the fact that the applied \(n_\textrm{max}\) was to low.
Calculations have been done also for elements of \(\textbf{V}\) generated with a logarithmic spread. Namely, each element was generated as \(s \times 10^b\) with \(s = \pm 1\) randomly assigned and exponent b drawn from the uniform distribution between \(-10\) and 0. With \(n_\textrm{max} = 10\), it was found that the percentage of convergence was lower than in the case of matrix elements drawn from the uniform distribution. In other calculations, a certain number of matrix elements was randomly set equal to zero to inspect the effect of the sparsity of \(\textbf{V}\). The effect was, again, a decrease of the percentage of convergence.
References
L.S.J. Allen, A primer on stochastic epidemic models: formulation, numerical simulation, and analysis. Infect. Dis. Model. 2, 128–142 (2017)
D. Frezzato, Sensitivity analysis of the reaction occurrence and recurrence times in steady-state biochemical networks. Math. Biosci. 332, 108518 (2021)
D. Schnoerr, G. Sanguinetti, R. Grima, Approximation and inference methods for stochastic biochemical kinetics—a tutorial review. J. Phys. A 50, 093001 (2017)
D.T. Gillespie, Stochastic simulation of chemical kinetics. Annu. Rev. Phys. Chem. 58, 35–55 (2007)
G.J. Moro, A. Ferrarini, A. Polimeno, P.L. Nordio, Models of conformational dynamics. In: Reactive and Flexible Molecules in Liquids. Kluwer Academinc Publishers, Dordrecht, pp. 107–109 (1989)
I.V. Gopich, A. Szabo, Theory of the statistics of kinetic transitions with application to single-molecule enzyme catalysis. J. Chem. Phys. 124, 154712 (2006)
S.C. Kou, B.J. Cherayil, W. Min, B.P. English, X.S. Xie, Single-molecule Michaelis-Menten equations. J. Phys. Chem. B 109, 19068–19081 (2005)
D. Loutchko, D. Gonze, A.S. Mikhailov, Single-molecule stochastic analysis of channeling enzyme tryptophan synthase. J. Phys. Chem. B 120, 2179–2186 (2016)
M.J. Schnitzer, S.M. Block, Statistical kinetics of processive enzymes. Cold Spring Harbor Symposia Quant. Biol. LX, 793–802 (1995)
A.B. Kolomeisky, M.E. Fisher, Molecular motors: a theorist’s perspective. Annu. Rev. Phys. Chem. 58, 675–695 (2007)
M.L. Mugnai, C. Hyeon, M. Hinczewski, D. Thirumalai, Theoretical perspectives on biological machines. Rev. Mod. Phys. 92, 025001 (2020)
S. Bai, D. Zhou, M.J. Davis, R.T. Skodje, Sum over histories representation of chemical kinetics. J. Phys. Chem. Lett. 6, 183–188 (2015)
A. Sabatino, D. Frezzato, Tagged-moiety viewpoint of chemical reaction networks. J. Chem. Phys. 150, 134104 (2019)
A. Sabatino, E. Penocchio, G. Ragazzon, A. Credi, D. Frezzato, Individual-molecule perspective analysis of chemical reaction networks: the case of a light-driven supramolecular pump. Angew. Chem. Int. Ed. 58, 14341–14348 (2019)
D. Frezzato, Stationary Markov jump processes in terms of average transition times: setup and some inequalities of kinetic and thermodynamic kind. J. Phys. A 53, 365003 (2020)
D. Frezzato, Dissipation-recurrence inequalities at the steady state. Phys. Rev. E 103, 032112 (2021)
P. Pietzonka, F. Ritort, U. Seifert, Finite-time generalization of the thermodynamic uncertainty relation. Phys. Rev. E 96, 012101 (2017)
J.M. Horowitz, T.R. Gingrich, Proof of the finite-time thermodynamic uncertainty relation for steady-state currents. Phys. Rev. E 96, 020103 (2017)
A.C. Barato, U. Seifert, Universal bound on the Fano factor in enzyme kinetics. J. Phys. Chem. B 119, 6555–6561 (2015)
A.C. Barato, U. Seifert, Thermodynamic uncertainty relation for biomolecular processes. Phys. Rev. Lett. 114, 158101 (2015)
Y. Song, C. Hyeon, Thermodynamic uncertainty relation to assess biological processes. J. Chem. Phys. 154, 130901 (2021)
R. Rao, L. Peliti, Thermodynamics of accuracy in kinetic proofreading: dissipation and efficiency trade-off. J. Stat. Mech. 06001 (2015)
K. Banerjee, A.B. Kolomeisky, O.A. Igoshin, Elucidation interplay of speed and accuracy in biological error correction. Proc. Natl. Acad. Sci. USA 114(26), 5183–5188 (2017)
J.D. Mallory, A.B. Kolomeisky, O.A. Igoshin, Trade-offs between error, speed, noise, and energy dissipation in biological processes with proofreading. J. Phys. Chem. B 123, 4718–4725 (2019)
Y. Song, C. Hyeon, Thermodynamic cost, speed, fluctuations, and error reduction of biological copy machines. J. Phys. Chem. Lett. 11, 3136–3143 (2020)
W.D. Piñeros, T. Tlusty, Kinetic proofreading and the limits of thermodynamic uncertainty. Phys. Rev. E 101, 022415 (2020)
M.M. Lin, Circuit reduction of heterogeneous nonequilibrium systems. Phys. Rev. Lett. 125, 218101 (2020)
N.G. van Kampen, Stochastic Processes in Physics and Chemistry (North-Holland, Amsterdam, 1992)
J. Schnakenberg, Network theory of microscopic and macroscopic behavior of master equation systems. Rev. Mod. Phys. 48, 571–585 (1976)
P. Nicolini, D. Frezzato, Features in chemical kinetics. I. Signatures of self-emerging dimensional reduction from a general formal of the evolution law. J. Chem. Phys. 138, 234101 (2013)
P. Nicolini, D. Frezzato, Features in chemical kinetics. II. A self-emerging definition of slow manifolds. J. Chem. Phys. 138, 234102 (2013)
A. Ceccato, P. Nicolini, D. Frezzato, Features in chemical kinetics. III. Attracting subspaces in a hyper-spherical representation of the reactive system. J. Chem. Phys. 143, 224109 (2015)
A. Ceccato, P. Nicolini, D. Frezzato, A low-computational-cost strategy to localize points in the slow manifold proximity for isothermal chemical kinetics. Int. J. Chem. Kinet. 49, 477–493 (2017)
A. Ceccato, P. Nicolini, D. Frezzato, Recasting the mass-action rate equations of open chemical reaction networks into a universal quadratic format. J. Math. Chem. 57, 1001–10018 (2019)
A. Ceccato, P. Nicolini, D. Frezzato, Attracting subspaces in a hyper-spherical representation of autonomous dynamical systems. J. Math. Phys. 58(9), 092701 (2017)
D. Loutchko, D. Gonze, A.S. Mikhailov, Single-Molecule stochastic analysis of channeling enzyme tryptophan synthase. J. Phys. Chem. B 120, 2179–2186 (2016)
J. Gunawardena, A linear framework for time-scale separation in nonlinear biochemical systems. PloS ONE 7, 36321 (2012)
M. Peschel, W. Mende, The Predator-prey Model: do We Live in a Volterra World? (Springer, New York, 1986)
J. Tóth, A.L. Nagy, D. Papp, The induced kinetic differential equation, in Reaction Kinetics: Exercises Programs and Theorems (Springer, New York, 2018)
B. Hernández-Bermejo, V. Fairén, Nonpolynomial vector fields under the Lotka-Volterra normal form. Phys. Lett. A 206, 31–37 (1995)
L. Brenig, A. Goriely, Universal canonical forms for time-continuous dynamical systems. Phys. Rev. A 40, 4119–4122 (1989)
J.L. Gouzé, Transformation of polynomial differential systems in the positive orthant. Technical report, INRIA, Sophia-Antipolis, 06561, Valbonne, France (1996)
V. Fairén, B. Hernández-Bermejo, Mass action law conjugate representation for general chemical mechanisms. J. Phys. Chem. 100, 19023–19028 (1996)
L. Brenig, Reducing nonlinear dynamical systems to canonical forms. Philos. Trans. R. Soc. A 376, 20170384 (2018)
B. Hernández-Bermejo, Stability conditions and Liapunov functions for quasi-polynomial systems. Appl. Math. Lett. 15, 25–28 (2002)
I.M. Gléria, A. Figueiredo, T.M.R. Filho, Stability properties of a general class of nonlinear dynamical systems. J. Phys. A 34(17), 3561–3575 (2001)
T.M.R. Filho, I.M. Gléria, A. Figueiredo, L. Brenig, The Lotka-Volterra canonical format. Ecol. Model. 183, 95–106 (2005)
M. Motee, B. Bahmieh, M. Khammash, Stability analysis of quasi-polynomial dynamical systems with applications to biological network models. Automatica 48, 2945–2950 (2012)
G. Szederkényi, A. Magyar, K.M. Hangos, Analysis and Control of Polynomial Dynamic Models with Biological Applications (Academic Press, New York, 2018)
I. Gléria, L. Brenig, T.M.R. Filho, A. Figueiredo, Permanence and boundedness of solutions of quasi-polynomial systems. Phys. Lett. A 381, 2149–2152 (2017)
A. Magyar, G. Szederkényi, K.M. Hangos, Globally stabilizing feedback control of process systems in generalized Lotka-Volterra form. J. Process Control 18, 80–91 (2008)
A. Magyar, K.M. Hangos, Globally stabilizing state feedback control design forLotka-Volterra systems based on underlying linear dynamics. IFAC-PapersOnLine 48, 1000–1005 (2015)
A.N. Gorban (editors), Coping with Complexity: Model Reduction and Data Analysis (Springer, Berlin, 2011)
K.M. Page, M.A. Nowak, Unifying evolutionary dynamics. J. Theor. Biol. 219, 93–98 (2002)
M. Smerlak, Quasi-species evolution maximizes genotypic reproductive value (not fitness or flatness). J. Theor. Biol. 522, 110699 (2021)
P.N. Brown, G.D. Byrne, A.C. Hindmarsh, VODE: a variable-coefficient ODE solver. SIAM J. Sci. Stat. Comput. 10(17), 1038–1051 (1989)
Acknowledgements
The author thanks Paolo Nicolini and Alessandro Ceccato for a former long standing and exciting collaboration on closely related topics.
Funding
No funding was received for conducting this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author has no competing interests to declare, and no financial and non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Appendix A: Asymptotic behaviour of \(\tilde{\textbf{W}}_n\) with matrices \(\textbf{V}\) randomly generated
Appendix A: Asymptotic behaviour of \(\tilde{\textbf{W}}_n\) with matrices \(\textbf{V}\) randomly generated
An empirical investigation has been conducted on the behavior of \(\tilde{\textbf{W}}_n\) for matrices \(\textbf{V}\) filled at random. In agreement with the analysis made in Ref. [30], it has been found that the convergence to a stable limit matrix \(\tilde{\textbf{W}}_\infty \) is indeed frequent, although the relationship between convergence and requisites about the entries of \(\textbf{V}\) is still missing.
For instance, if \(\textbf{V}\) is fully filled with entries between -1 and +1 randomly drawn from the uniform distribution, it was estimated that the convergence occurs in at least 15% of the cases for \(d=3\), 8% of the cases for \(d=4\), and 5% for \(d=5\).Footnote 9 The percentage of convergence of course changes if the matrix elements are generated in different ways, and it is found to depend also on the sparsity of \(\textbf{V}\).Footnote 10 The “missed convergence” mostly manifested in an even-odd alternation of the \(\tilde{\textbf{W}}_n\) between two limit forms \(\tilde{\textbf{W}}_\infty \) and \(-\tilde{\textbf{W}}_\infty \). In addition it emerged, at least for the low d cases which could be directly inspected, that \(\tilde{\textbf{W}}_\infty \) typically (but likely not exclusively) tends to a dyadic form in which all rows are given by one row multiplied by a certain factor.
As an example of convergence to \(\tilde{\textbf{W}}_\infty \), let us consider the following matrix randomly generated:
Its eigenvalues are \(-0.9763\), \((0.7514 \pm \imath \, 0.2316)\). The matrices \(\tilde{\textbf{W}}_{15}\) and \(\tilde{\textbf{W}}_{14}\) are
The convergence to \(\tilde{\textbf{W}}_\infty \) is practically already achieved and, as can be seen, the resulting matrix has a dyadic form: the second and the third rows tend to be identical, while the first row tends to be equal to these rows multiplied by a factor \(\simeq -1.54\). Here is another example:
which gives
In this case, the eigenvalues of \(\textbf{V}\) are all real and equal to 1.1617, \(-0.8483\), \(-0.5298\). The convergence to \(\tilde{\textbf{W}}_\infty \) is practically achieved and the limit matrix is again a dyadic product in which the second row and the third row are, respectively, about 0.013 and 0.56 times the first row.
Based on the direct observations with d from 3 to 5, the conjecture made here is that the convergence to a fixed matrix \(\tilde{\textbf{W}}_\infty \) is encountered for any dimensions d provided that the matrix \(\textbf{V}\) fulfills some requisites that are however still unknown. In particular, there is no evident relationship between convergence to \(\tilde{\textbf{W}}_\infty \) and eigenvalues of the matrix \(\textbf{V}\): the convergence has been observed either when the eigenvalues are all real or when some are complex-valued, and even regardless of the sign of the real parts. Concerning the convergence rate (versus n) to \(\tilde{\textbf{W}}_\infty \), generally it decreases as d increases and depends on the sparsity of \(\textbf{V}\) in a way not yet systematically characterized.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Frezzato, D. Intrinsic timing in classical master equation dynamics from an extended quadratic format of the evolution law. J Math Chem 61, 806–834 (2023). https://doi.org/10.1007/s10910-022-01435-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10910-022-01435-7
Keywords
- Master equation
- Markov jump processes
- Quadratization of ordinary differential equations
- Embedding into Lotka–Volterra