Spectral theorem for convex monotone homogeneous maps, and ergodic control

https://doi.org/10.1016/S0362-546X(02)00170-0Get rights and content

Abstract

We consider convex maps f:RnRn that are monotone (i.e., that preserve the product ordering of Rn), and nonexpansive for the sup-norm. This includes convex monotone maps that are additively homogeneous (i.e., that commute with the addition of constants). We show that the fixed point set of f, when it is nonempty, is isomorphic to a convex inf-subsemilattice of Rn, whose dimension is at most equal to the number of strongly connected components of a critical graph defined from the tangent affine maps of f. This yields in particular an uniqueness result for the bias vector of ergodic control problems. This generalizes results obtained previously by Lanery, Romanovsky, and Schweitzer and Federgruen, for ergodic control problems with finite state and action spaces, which correspond to the special case of piecewise affine maps f. We also show that the length of periodic orbits of f is bounded by the cyclicity of its critical graph, which implies that the possible orbit lengths of f are exactly the orders of elements of the symmetric group on n letters.

Introduction

We say that a map f:RnRn is monotone if for all x,y∈Rn, xyf(x)⩽f(y), where ⩽ denotes the product ordering of Rn (xy if xiyi, for all 1⩽in). We say that f is additively homogeneous if for all λ∈R, x∈Rn, f(λ+x)=λ+f(x), where λ+x=(λ+x1,…,λ+xn). It is easy to see that a monotone homogeneous map is nonexpansive for the sup-norm: for all x,y∈Rn, |f(x)−f(y)|⩽|xy|, where |x|=max1⩽i⩽n|xi| (see [17]).

Monotone homogeneous maps arise classically in optimal control and game theory (see for instance [10], [31], [71], [32], [20], [61]), in the modeling of discrete events systems (see [6], [25], [14], [15], [68], [21], [26], [27]), and in nonlinear potential theory [19], as nonlinear extension of Markov transitions. They also arise in nonlinear Perron–Frobenius theory, when one considers multiplicatively homogeneous maps F acting on a cone and preserving the order of the cone: in the simplest case, when the cone is (R+)n (where R+={x∈R|x>0}), the transformation F↦log∘F∘exp (where log:(R+)nRn is the map which does log entrywise, and exp=log−1) sends the set of monotone multiplicatively homogeneous maps to the set of monotone additively homogeneous maps. See for instance [11], [38], [44], [41], [46], [47], [62], [72] for various studies and applications.

A basic problem, for a monotone homogeneous map f, is the existence, and uniqueness (up to an additive constant), of the additive eigenvectors of f, which are the v∈Rn such that f(v)=λ+v, for some additive eigenvalue λ∈R. In the sequel, we will omit the term “additive”, when the additive nature of the objects will be clear from the context. When f has an eigenvector v with eigenvalue λ, by homogeneity of f, fk(v)=+v holds for all k⩾0, and by nonexpansiveness of f, |fk(x)−v|=|fk(x)−fk(v)|⩽|xv|, hence,fk(x)=kλ+O(1)whenk→∞for all x∈Rn (all the orbits of f have a linear growth rate of λ). This implies in particular that the eigenvalue λ is unique. Hence, we can speak without ambiguity of the eigenspace of f, which is the set E(f)={x∈Rn|f(x)=λ+x}. In many applications, the eigenvalue and eigenvector are fundamental objects: for instance, in stochastic control, the eigenvalue gives the optimal reward per time unit, and eigenvectors give stationary rewards (we explain this in detail in Section 7). In discrete event systems applications, the eigenvalue gives the throughput, and eigenvectors give stationary schedules.

Several Perron–Frobenius like theorems guarantee the existence of eigenvectors of monotone (additively) homogeneous maps RnRn. Such results go back at least to Kreı̆n and Rutman [34, Section 7], in the context of monotone multiplicatively homogeneous maps leaving a cone in a Banach space invariant, and to Morishima, whose book [44] contains a complete study of finite dimensional nonlinear Perron–Frobenius theory. A modern overview appears in the memoirs of Nussbaum [46], [47], which contain in particular general existence results for eigenvectors, following [45]. Different existence conditions appeared in [53]. General results on the geometry of the eigenspaces are available, for instance, the result of Bruck [12] shows in particular that E(f) is the image of a nonexpansive projector and a fortiori is connected, see also [46, Theorems 4.5–4.7].

In this paper, we describe the eigenspaces of convex monotone homogeneous maps f:RnRn. (We say that a Rn-valued map is convex when its coordinates are convex. We refer the reader to [57] for all convexity notions used in the paper: subdifferentials, domain, Fenchel transform, etc.)

To state our main result, we need a few definitions (see Section 2 for details). We first generalize the notion of subdifferential to maps RnRn by setting, for x∈Rn, ∂f(x)={P∈Rn×n|f(y)−f(x)⩾P(y−x),∀y∈Rn}. It is easy to see (Corollary 2.2 and Eq. (4) below) that by monotonicity and homogeneity of f, the elements of ∂f(x) are stochastic matrices. If v is an eigenvector of f, we call critical graph of f, the (directed) graph Gc(f) which is the union of final graphs of stochastic matrices P∂f(v) (we call final graph of a stochastic matrix the restriction of its graph to the set of final classes, see 2.2 Convex rectangular sets of stochastic matrices, 2.3 Critical graph of convex monotone homogeneous maps). The graph Gc(f) is independent of the choice of the eigenvector v (Proposition 2.5 below). We call critical nodes of f, the nodes of Gc(f), and denote by Nc(f) the set of critical nodes. We call critical classes of f the sets of nodes C1,…,Cs of the strongly connected components of Gc(f), G1,…,Gs. We call cyclicity of Gi, and denote by c(Gi), the gcd of the lengths of the circuits of Gi, and we define the cyclicity of f by c(f)=lcm(c(G1),…,c(Gs)). We say that a monotone homogeneous map g:U⊂Rn→V⊂Rp is a monotone homogeneous isomorphism if it has a monotone homogeneous inverse. The following theorem (already announced in [2]), gathers results from Theorem 3.4, Corollary 3.6, Corollary 5.7, and Theorem 6.6 below.

Theorem 1.1 Convex spectral theorem

Let f:RnRn denote a convex monotone homogeneous map that has an eigenvector. Denote by C=Nc(f) the set of critical nodes of f,c=c(f) the cyclicity of f, and λ the unique eigenvalue of f. Then,

  • (1)

    the restriction r:RnRC,x↦(xi)i∈C, is a monotone homogeneous isomorphism from E(f) to its image Ec(f);

  • (2)

    Ec(f) is an inf-subsemilattice of (RC,⩽);

  • (3)

    Ec(f) is a convex set whose dimension is at most equal to the number of critical classes of f, and this bound is attained when f is piecewise affine;

  • (4)

    for all x∈Rn, fkc(x)−kcλ has a limit when k→∞.

In particular, when f has only one critical class, the eigenvector of f is unique (up to an additive constant). It also follows from the last assertion of the theorem that the set of limit points of fk(x)− when k→∞, and x∈Rn, is precisely E(fc). Theorem 1.1 also allows us to bound the dimension of this set. Indeed, we shall see in Theorem 4.1 and Proposition 5.3 below that the set of critical nodes is the same for f and fc, and that fc has c(G1)+⋯+c(Gs) critical classes. Hence, applying Theorem 1.1 to fc, we get that the restriction r is a monotone homogeneous isomorphism from E(fc) to a convex set, Ec(fc), of dimension at most c(G1)+⋯+c(Gs), the bound being attained when f is piecewise affine.

The paper is devoted to the proof (2 Class structure of convex monotone homogeneous maps, 3 Structure of eigenspaces, 4 Critical graph of, 5 Cyclicity theorem for convex monotone homogeneous maps, 6 Piecewise affine convex monotone homogeneous maps) and to the stochastic control interpretation (Section 7) of Theorem 1.1. In Section 2, we detail the definitions and properties of subdifferentials and critical graph of convex monotone homogeneous maps. An important element of the proofs is the maximum principle for Markov chains (Lemma 2.9). In Section 3, we establish the first part of Theorem 1.1 concerning the structure of the eigenspace: points 1, 2 and the first assertion in point 3. The main argument is again the maximum principle. Section 4 is devoted to further tools and properties used in the remaining sections, which are of independent interest: Theorem 4.1 shows that Gc(fk)=Gc(f)k (this will be used in Section 5 for the proof of the cyclicity part of Theorem 1.1); we also introduce directional derivatives (which will be used in Section 6 to connect E(f) to E(fv′) for any eigenvector v), additive recession functions (formula (15)), invariant critical classes and the associated decomposition of f (Lemma 4.9), and also a characterization of the set of critical nodes in terms of supports of nonlinear “excessive” measures (Proposition 4.5).

Section 5 is devoted to the proof of point 4 of Theorem 1.1. This result relies on a more general theorem of Nussbaum [48] and Sine [67], which states that if f:RnRn is nonexpansive for the sup-norm and has a fixed point, then, for all x∈Rn, fkc(x) converges when k→∞, for some minimal constant c which can be bounded by a function of n. When f is convex monotone and homogeneous, the last assertion of the convex spectral theorem shows that the possible values of c are exactly the orders of elements of the symmetric group on n letters. Equivalently, convex monotone homogeneous maps have the same orbit lengths as permutation matrices, a result which was known to be true in the special cases of linear maps associated to nonnegative matrices (see [52, Chapter 9]), of linear maps over the max-plus semiring (see [13], [49]), and also of piecewise affine convex monotone homogeneous maps (which include max-plus linear maps), see the discussion in Section 1.2 below. More generally, computing the orbit lengths of nonexpansive maps for polyhedral norms raises interesting combinatorial and analytical problems (see in particular [1], [70], [63], [64], [48], [50], [51], [52], [36]).

The equality in point 3 of Theorem 1.1 is proved in Section 6. As will be discussed in Section 1.2, this part of the theorem has already been proved by Romanovsky [60] and by Schweitzer and Federgruen [66]. We provide here an independent proof, which emphasizes the qualitative properties of E(f), using the tools of Section 4. We also give a polynomial time algorithm to compute Gc(f).

Convex monotone homogeneous maps f:RnRn are exactly dynamic programming operators associated to stochastic control problems with state space {1,…,n}. Computing the stationary solutions and the asymptotic behavior of solutions of dynamic programming equations is an old problem of stochastic control which is essentially equivalent to that of computing the eigenspace E(f) and the asymptotics of fk when k→∞. This has been much studied in the stochastic control literature, particularly in the case of finite action spaces, which corresponds to piecewise affine maps. In this special case, results equivalent to the third assertion of Theorem 1.1 were obtained by Romanovsky [60] using linear programming techniques, and also by Schweitzer and Federgruen [66] who gave an explicit representation of E(f) in terms of resolvents associated to optimal strategies (see [66, Theorem 4.1]). Again in this special case, a result equivalent to the fourth assertion of Theorem 1.1 was stated by Lanery [35]. The arguments of [35] only proved the special case where Nc(f)={1,…,n}, see the discussion in the introduction and in Note 1 of [65]. A proof valid for a general Nc(f)⊂{1,…,n} was given by Schweitzer and Federgruen [65], who also proved the optimality of c(f).

The special case of deterministic control problems leads to maps f that are max-plus linear. These maps have been studied independently by the max-plus community. In this context, the dimension of the eigenspace was characterized by Gondran and Minoux [24], and the remaining part of the max-plus spectral theorem, dealing with cyclicity, was obtained by Cohen et al., [13] (see also [6]). (Note however that more precise results—explicit form of the eigenspace, finite time convergence of the iterates—are available in the max-plus case.) The max-plus spectral theorem has a long story, which goes back to Cuninghame–Green (see [18] and the references therein), Romanovsky [59], and Vorobyev [69], to quote the most ancient contributions. See the collection of articles [40], Kolokoltsov and Maslov [33], and the references therein, for generalizations to infinite dimension. See also [23], [7] for surveys.

The present work was inspired by the max-plus spectral theorem and uses nonexpansive maps techniques (we were unaware of the results of [35], [60], [65], [66]). We next emphasize differences with earlier results. We consider general convex monotone homogeneous maps, which correspond to stochastic control problems with finite state space and arbitrary action spaces, whereas the results in [35], [60], [65], [66] require the action space to be finite. Our proof technique, which relies on the maximum principle, can be naturally transposed to other (infinite dimensional) contexts. Another tool in our proof is the critical graph Gc(f), which generalizes the critical graph that appears in max-plus algebra (see Proposition 2.7). The critical graph already appeared in [60, p. 491], with a different definition in terms of optimal policies. The new definition that we give here in terms of subdifferentials leads in particular to a polynomial time algorithm to compute Gc(f) (see Section 6.3). (The equivalence of both definitions is shown in Proposition 7.2.) It should also be noted that when passing from the case of finite action spaces to arbitrary action spaces, new phenomena occur. For instance, Example 3.9 shows that we cannot hope, in general, to characterize the dimension of eigenspaces in terms of graphs like Gc(f).

Let us mention in passing that the critical graph has an intuitive interpretation in terms of “recurrence”. For a Markov chain, a node is recurrent if the probability of return to this node is equal to one. For a max-plus matrix with eigenvalue 0, a node is “recurrent”, i.e. belong to a critical class, if we can return to this node with zero reward. When f is a convex monotone homogeneous map with eigenvalue 0, a node i is “recurrent”, i.e. belong to a critical class, if we can find a strategy for which, starting from i, we eventually return to i with probability 1 and zero mean reward. This provides a new illustration of the analogy between probability and optimization developed in [39, Chapter VIII], [43], [55], [3], [42], [56, Section 4.2.], [37], [54].

Ergodic control problems of diffusion processes lead to spectral problems for infinite dimensional monotone homogeneous semigroups which can be expressed in terms of ergodic Hamilton–Jacobi–Bellman (HJB) partial differential equations. In [8], Bensoussan proved uniqueness of the eigenvector (as weak solution of the ergodic HJB equation) under assumptions, which translated in finite dimension imply irreducibility of stochastic matrices P∂f(v). Inspired by the results of the present paper, the first author, Sulem and Taksar [4] proved uniqueness of the viscosity solution of a special ergodic HJB equation. This yields an example of concrete situation where some nonoptimal stationary strategies have several final classes, whereas the optimal ones have only one final class (translated to our setting, this means that for some x∈Rn and P∂f(x), P may have several final classes, whereas there exists an eigenvector v such that all elements of ∂f(v) have one final class).

The uniqueness result that follows from Theorem 1.1 can be thought of as a partial extension of the condition of Nussbaum [46, Theorem 2.5]: specialized to convex monotone homogeneous maps f:RnRn, the result of Nussbaum shows that if f is C1 and for all eigenvectors v, f′(v) has only one final class (in this case of course f has a unique critical class), then the eigenvector of f is unique. The idea of all these results is that the dimension of E(f) can be bounded by looking at “linearizations” of f near an eigenvector.

It is instructive to note that the uniqueness of eigenvectors in Rn is governed by the same kind of graph properties as the existence of eigenvectors, albeit the graphs are different. For instance, a result of the second author and Gunawardena [22, Theorem 2] guarantees the existence of an eigenvector for a monotone homogeneous map which has a strongly connected graph. Here, the graph G(f) of a monotone homogeneous map f:RnRn has nodes {1,…,n} and an arc ij if limν→∞fi(νej)=+∞, where ej denotes the jth vector of the canonical basis of Rn. Another way to guarantee the existence of an eigenvector is to use the convex spectral theorem itself, thanks to the following observation taken from [22]. We denote by f̂(x)=limμ→∞μ−1f(μx) the recession function of f ( need not exist when f is monotone and homogeneous, but it does exist when f is convex). We have f̂(0)=0, and when f is (additively) homogeneous, so does , so that all points on the diagonal are (trivial) fixed points of f. It is proved in [22] that if the recession function of a monotone homogeneous map f exists and has only fixed points on the diagonal, then, f has an eigenvector. Combining this observation with the convex spectral theorem, we obtain:

Corollary 1.2

A monotone homogeneous map has an eigenvector if its recession function exists, is convex, and has only one critical class.

If f is a convex monotone homogeneous map, it is not difficult to see that the recession function is exactly the support function of the domain of the Fenchel transform f of f (defined in Section 2.1 below), that is f̂(x)=supP∈domfPx. In this formula, one can replace domf by its closure cl(domf), which is equal to f̂(0). The graph G(f) is the union of the graphs of P∈domf, or equivalently the union of the graphs of P∈cl(domf), whereas the critical graph Gc(f̂) is the union of the final graphs of P∈cl(domf). If G(f) is strongly connected, one can see that Gc(f̂) is also strongly connected, so that in the special case of convex monotone homogeneous maps, Corollary 1.2 is stronger than Theorem 2 of [22] (which however holds in a more general context).

Finally, let us mention two immediate extensions of the convex spectral theorem. First, since the map f↦(x↦−f(−x)) sends convex monotone homogeneous maps to concave monotone homogeneous maps, there is of course a dual concave spectral theorem. Another, more interesting, extension, is obtained by considering subhomogeneous maps f, which satisfy f(λ+x)⩽λ+f(x), for all λ⩾0 and x∈Rn. It is easy to see that a monotone map is subhomogeneous if, and only if, it is nonexpansive for the sup-norm. To a monotone subhomogeneous map f:RnRn, we associate canonically a monotone homogeneous map g:Rn+1Rn+1,g(x,y)=y+f(−y+x)y,∀x∈Rn,y∈R(this is a nonlinear extension of the classical way of passing from a substochastic to a stochastic matrix, by adding a cemetery state, in this nonlinear context, this construction is due to Gunawardena and Keane [28]). A vector z∈Rn is a fixed point of f, if, and only if, (z,0) is an eigenvector of g (and the eigenvalue is 0). Using this construction, one translates readily the convex spectral theorem to a theorem describing fixed point sets and the asymptotics of the iterates of convex monotone subhomogeneous maps. (We might also use this construction, with −λ+f instead of f, to describe the eigenspace of f for an additive eigenvalue λ, but when f is only monotone and subhomogeneous, λ need not be unique, and it tells little about the asymptotics of fk, in general.) For a convex monotone subhomogeneous map f with fixed point v, the critical graph Gc(f) of f is defined as the union of the graphs of the matrices PCC, where P∂f(v), C is a final class of P, and the C×C submatrix of P, PCC, is stochastic (when f is homogeneous, this property is automatically satisfied). Equivalently, Gc(f) (which can be empty) is the restriction of Gc(g) to {1,…,n}. The notions of critical classes and cyclicity are defined from Gc(f) as above. When Gc(f) is empty, we have Nc(f)=∅, and we take the convention R={0}, and c(f)=1.

Corollary 1.3

Let f:RnRn denote a convex monotone subhomogeneous map that has a fixed point. Then, all the conclusions of the convex spectral theorem apply to f and λ=0. In particular, if f has no critical classes, then its fixed point is unique.

Section snippets

Subdifferentials of convex monotone homogeneous maps

We shall first consider scalar monotone homogeneous maps g:RnR (which satisfy xyg(x)⩽g(y) for all x,y∈Rn, and g(λ+x)=λ+g(x), for all λ∈R and x∈Rn). The Fenchel transform of g is the map g:RnR∪{+∞}, g(p)=supx∈Rn(p·x−g(x)). We denote by domg={p∈Rn|g(p)<∞} the domain of g, and by Sn={p∈Rn|1⩽i⩽npi=1,p1,…,pn⩾0} the set of stochastic vectors.

Proposition 2.1

If g:RnR is monotone and homogeneous, then, domg is included in Sn.

Proof

Let 1n denote the vector of Rn whose entries are all equal to 1. If g is

Structure of eigenspaces

In order to make more apparent the proof idea, we first show a simple result.

Theorem 3.1

The eigenvector of a convex monotone homogeneous map with a unique critical class is unique, up to an additive constant.

Proof

Let C denote the critical class of f, and let v,v′ be two eigenvectors of f. Using Assertion 1 of Corollary 2.6, we get matrices P∂f(v) and P′∈∂f(v′) such that C is the unique final class of P and P′. Since P∂f(v), v′−v=f(v′)−f(v)⩾P(v′−v), hence, by Lemma 2.9, v′−v is constant on C, and it attains

Critical graph of fk

In this section, we establish the following result which is central in the proof of the cyclicity theorem. If G is a graph, we call Gk the graph with same nodes as G and arcs (ij) when there is a directed path ii1→⋯→ik=j with length k in G.

Theorem 4.1

Let f denote a convex monotone homogeneous map RnRn that has an eigenvector. Then, for all k⩾1, Gc(fk)=Gc(f)k, in particular Nc(fk)=Nc(f).

The proof of Theorem 4.1 needs tools and results, of independent interest, which involve one-sided directional

Cyclicity theorem for convex monotone homogeneous maps

In this section, we use our knowledge of the eigenspace of f to study the asymptotic behavior of fk when k tends to +∞. In particular, we are interested in the periodic orbits of f, which are of the form {fk(x)}k∈N, with fc(x)=x for some c⩾1. The set of such c is exactly the set of multiples of a positive integer, which is the length of the orbit {fk(x)}k∈N.

Let us first recall some more or less classical facts on periodic orbits of stochastic matrices. The cyclicity c(G) of a strongly connected

Dimension of the eigenspace

As discussed in Section 1.2, the dimension of the eigenspace of a piecewise affine convex monotone homogeneous map was characterized by Romanovsky [60] and by Schweitzer and Federgruen [66]: this shows that the bound on the dimension given in Corollary 3.6, is attained when f is piecewise affine. In this subsection, we give an independent proof of this fact, which shows some qualitative properties of E(f) (connection between E(f) and E(fv′), for any eigenvector v, role of invariant critical

Stochastic control interpretation

In this section, we briefly explain how the above results can be applied to stochastic control. This application also makes the results more intuitive. See for instance [71], or [29] for more background on stochastic control.

A Markov control model with state space {1,…,n} is a 4-uple (A,{Ai}1⩽in,{ri}1⩽in,{Pi}1⩽in), where: A is a set, called action space; for each state 1⩽in, Ai is a nonempty subset of A, whose elements are interpreted as possible actions; ri is a map from Ai to R, the image

Acknowledgements

The authors thank J. Gunawardena, J.P. Quadrat, and C. Sparrow, for many useful discussions.

References (73)

  • F. Baccelli et al.

    Synchronization and Linearity: An Algebra for Discrete Events Systems

    (1992)
  • A. Bensoussan

    Perturbation Methods in Optimal Control

    (1988)
  • A. Berman et al.

    Nonnegative matrices in the mathematical sciences, Classics in Applied Mathematics

    (1994)
  • T. Bewley et al.

    The asymptotic solution of a recursion equation occurring in stochastic games

    Math. Oper. Res.

    (1976)
  • P. Bougerol

    Almost sure stabilizability and Riccati's equation of linear systems with random parameters

    SIAM J. Control Optim.

    (1995)
  • R.E. Bruck

    Properties of fixed-point sets of nonexpansive mappings in Banach spaces

    Trans. Amer. Math. Soc.

    (1973)
  • G. Cohen, D. Dubois, J.-P. Quadrat, M. Viot, Analyse du comportement périodique des systèmes de production par la...
  • G. Cohen, S. Gaubert, J.-P. Quadrat, Algebraic system analysis of timed Petri nets, in: J. Gunawardena (Ed.),...
  • G. Cohen, S. Gaubert, J.-P. Quadrat, Asymptotic throughput of continuous timed petri nets, in: Proceedings of the 34th...
  • G. Cohen, S. Gaubert, J.-P. Quadrat, Linear projectors in the max-plus algebra, in: Proceedings of the IEEE...
  • M.G. Crandall et al.

    Some relations between nonexpansive and order preserving maps

    Proc. AMS

    (1980)
  • R. Cuninghame-Green

    Minimax Algebra

    (1979)
  • C. Dellacherie

    Théorie générale du potentiel

    I, Astérisque

    (1996)
  • J.A. Filar et al.

    Competitive Markov Decision Processes

    (1997)
  • S. Gaubert, J. Gunawardena, A non-linear hierarchy for discrete event dynamical systems, in: Proceedings of the Fourth...
  • S. Gaubert, J. Gunawardena, The Perron–Frobenius theorem for homogeneous, monotone functions, Hewlett-Packard Technical...
  • S. Gaubert, M. Plus, Methods and applications of (max,+) linear algebra, in: Proceedings of STACS’97, Rapport de...
  • M. Gondran, M. Minoux, Valeurs propres et vecteurs propres dans les dioı̈des et leur interprétation en théorie des...
  • J. Gunawardena

    Min–max functions

    Discrete Event Dyn. Systems

    (1994)
  • J. Gunawardena (Ed.), Idempotency, Publications of the Isaac Newton Institute, Cambridge University Press, Cambridge,...
  • J. Gunawardena, From max-plus algebra to nonexpansive maps: a nonlinear theory for discrete event systems, Theoret....
  • J. Gunawardena, M. Keane, On the existence of cycle times for some nonexpansive maps, BRIMS Technical Report...
  • O. Hernández-Lerma et al.

    Discrete-Time Markov Control Processes

    (1996)
  • J.G. Kemeny, J.L. Snell, A.W. Knapp, Denumerable Markov Chains, The University Series in Higher Mathematics, Van...
  • E. Kohlberg

    Invariant half-lines of nonexpansive piecewise-linear transformations

    Math. Oper. Res.

    (1980)
  • V.N. Kolokoltsov, On linear, additive and homogeneous operators in idempotent analysis, in: V.P. Maslov, S.N. Samborski...
  • Cited by (50)

    • A nonlinear Perron–Frobenius approach for stability and consensus of discrete-time multi-agent systems

      2020, Automatica
      Citation Excerpt :

      Sub-Homogeneity Order-preserving and sub-homogeneous maps arise in a variety of applications, including optimal control and game theory (Akian & Gaubert, 2003), mathematical biology (Rosenberg & Sorin, 2001), analysis of discrete event systems (Gunawardena, 2003) and so on. Concluding this section, we recall some basic concepts on periodic points which are instrumental to state our main results.

    • Generic uniqueness of the bias vector of finite zero-sum stochastic games with perfect information

      2018, Journal of Mathematical Analysis and Applications
      Citation Excerpt :

      In the weak KAM setting, the bias vector becomes the solution of an ergodic Hamilton–Jacobi PDE; Figalli and Rifford showed in [29] that this solution is unique for generic perturbations of a Hamiltonian by a potential function. In the stochastic case, the structure of the set of bias vectors is still known when the state space is finite, see Akian and Gaubert [3]. In the two-player case, the structure of the set of bias vectors is less well known, although the description of this set remains a fundamental issue.

    • A classification of invariant distributions and convergence of imprecise Markov chains

      2013, Linear Algebra and Its Applications
      Citation Excerpt :

      In Section 4 we finally use these results to prove our main results in Theorems 2 and 3. A similar construction is used by Akian and Gaubert [1] to construct an additive eigenvector of a convex monotone and additively homogeneous map. In the theory of Markov chains it is an important question which states are possible to be visited at following times when a state has been observed at given time.

    View all citing articles on Scopus

    This work was partially supported by the European Community Framework IV program through the research network ALAPEDES (“The Algebraic Approach to Performance Evaluation of Discrete Event Systems”).

    1

    This work has been performed when the second author was with École Nationale Supérieure des Techniques Avancées, Paris, France.

    View full text