Geometry of deformed exponential families: Invariant, dually-flat and conformal geometries

https://doi.org/10.1016/j.physa.2012.04.016Get rights and content

Abstract

An information-geometrical foundation is established for the deformed exponential families of probability distributions. Two different types of geometrical structures, an invariant geometry and a flat geometry, are given to a manifold of a deformed exponential family. The two different geometries provide respective quantities such as deformed free energies, entropies and divergences. The class belonging to both the invariant and flat geometries at the same time consists of exponential and mixture families. Theq-families are characterized from the viewpoint of the invariant and flat geometries. The q-exponential family is a unique class that has the invariant and flat geometries in the extended class of positive measures. Furthermore, it is the only class of which the Riemannian metric is conformally connected with the invariant Fisher metric.

Highlights

► Information geometries on a manifold of a deformed exponential family are studied. ► They define respective quantities such as deformed free energies and entropies. ► Geometry of the q-exponential family is characterized via invariance and flatness. ► Conformal structure of the flat geometry of the q-exponential family is studied.

Introduction

Since the introduction of q-entropy by Tsallis [1] (see also an extensive monograph [2]), much attention has been paid to non-extensive statistical mechanics. It is related to various ‘non-standard phenomena’ subject to the power law not only in statistical physics but in economics and disaster statistics. Here, families of probability distributions of the q-exponential family and more general deformed exponential families play a major role. In the present paper, a geometrical foundation is given to these families of distributions from the point of view of information geometry  [3].

The deformed exponential family was introduced and studied extensively by Naudts [4], [5] (see also a monograph [6]). Kaniadakis et al. [7] studied the κ-exponential family which belongs to the deformed exponential family. Its mathematical structure was studied by Pistone [8] and Vigelis and Cavalcante [9]. See other examples with interesting discussions [10], [11]. In statistics, a similar notion of a generalized exponential family [12] or the U-model [13], [14] is discussed on the bases of respective motives.

Many useful concepts such as generalized entropy, divergence and escort probability distribution have been proposed. However, their relationships have not necessarily been well understood theoretically and are waiting for further geometrical and statistical elucidation. It is also useful to characterize the q-families in the class of general deformed exponential families.

In the present study, information geometry [3] is used to give a foundation to the deformed exponential families. Two types of geometry can be introduced in the manifold of a deformed exponential family: One is the invariant geometry, where the Fisher information is the unique Riemannian metric (Chentsov [15]; also see Ref. [3]) together with a dual pair of invariant affine connections (α-connections). The other is the dually flat geometry [3] (also see Ref. [16]), which is not necessarily invariant but accompanies the Legendre structure. The escort probability distribution belongs to the latter geometry. The two geometries give different free-energies, entropies and divergences in general.

The exponential and mixture families are characterized by the property that they sit at the intersection of the classes of the invariant and flat geometries. The q-exponential family is then characterized from two viewpoints: One is invariance and flatness in the class of positive measures and the other is conformal geometry [17], [18]. It is shown that the q-family is a unique class of flat geometry that is connected conformally to the invariant geometry.

Section snippets

Deformed exponential family

We follow Naudts [6], [19] for the formulation of the deformed exponential family. Given a positive increasing function χ(s) on (0,)R, a deformed logarithm, called the χ-logarithm, is defined by lnχ(s)=1s1χ(t)dt. This is a concave monotonically increasing function. When χ is a power function, χ(s)=sq,q>0,(1) gives the q-logarithm lnq(s)=11q(s1q1). The ordinary logarithm is obtained by taking the limit of q=1.

The inverse of the χ-logarithm is the χ-exponential, given by expχ(t)=1+0tλ(s)ds,

α-geometry

A χ-family (7) is considered as a manifold with a (local) coordinate system θ. We use the invariance principle to introduce a unique geometrical structure (see Refs. [3], [15]).

Invariance principle: The geometry is invariant under the transformation of random variable x to y, provided y is a sufficient statistics.

The invariant geometry is characterized by the two tensors, gij(θ)=E[ilogp(x,θ)jlogp(x,θ)],Tijk(θ)=E[ilogp(x,θ)jlogp(x,θ)klogp(x,θ)] where i=/θi and E denotes expectation. The

Flat χ-geometry

Apart from the invariancy principle, we give a new dually flat structure to the χ-family, different from the invariant geometry. This is called the χ-geometry. When χ(s)=sq,q>0, we particularly call it the q-geometry. The χ-free-energy ψ(θ) of a χ-family is known to be a convex function [6]. We give its simple proof for later use. We have, by putting u(x)=expχ(x) for simplicity, ip(x,θ)=u(θxψ)(xiiψ),ijp(x,θ)=u(θxψ){(xiiψ)(xjjψ)}u(θxψ)ijψ. Since ip(x,θ)dx=ijp(x,θ)dx=0,

Conformal transformation connecting invariant and flat geometries

Two geometries, invariant and flat, induced in the χ-structure of Sn were studied in the previous sections. It is interesting to know how they are related. In order to answer this question, we calculate the χ-Fisher metric by using the χ-divergence.

Theorem 12

The χ-Fisher metric of Sn is given bygijχ=12hχ{v(pi)u(v(pi))δij+v(p0)u(v(p0))},i,j=1,,n.

Proof

The Taylor expansion of the χ-divergence is calculated as Dχ[p:p+dp]=1hχi=0nu{v(pi)}{v(pi)v(pi+dpi)}=12hχu{v(pi)}v(pi)dpi2. Hence, the χ-Fisher

Conclusions

The geometrical structures of the manifold of a general deformed exponential family were studied. The two geometrical structures were introduced, one from the viewpoint of invariance and the other from the viewpoint of flatness. They give different definitions of generalized free energy, entropy and divergence. The exponential and mixture families are characterized uniquely by the fact that the two geometries coincide. The q-exponential families are characterized by the fact that the two

References (32)

  • G. Pistone

    κ-exponential models from the geometrical viewpoint

    Eur. Phys. J. B

    (2009)
  • R. Vigelis et al.

    On the φ-exponential family of probability distributions

    J. Theor. Probab.

    (2011)
  • C. Beck

    Generalized information and entropy measures in physics

    Contemp. Phys.

    (2009)
  • P.D. Grunwald et al.

    Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory

    Ann. Stat.

    (2004)
  • S. Eguchi

    Information geometry and statistical pattern recognition

    Sugaku Expositions

    (2006)
  • Y. Fujimoto et al.

    A generalization of independence in naive Bayes model

    Lect. Notes Comput. Sci.

    (2010)
  • Cited by (71)

    • Informative fractal dimension associated with nonmetricity in information geometry

      2023, Physica A: Statistical Mechanics and its Applications
    • Independent Approximates enable closed-form estimation of heavy-tailed distributions

      2022, Physica A: Statistical Mechanics and its Applications
    • λ-Deformed probability families with subtractive and divisive normalizations

      2021, Handbook of Statistics
      Citation Excerpt :

      It eventually became clear in the 2018 paper Naudts and Zhang (2018) by Naudts and the first author that (i) the ϕ-model and U-model turned out to be equivalent; (ii) they are special cases of the (ρ, τ)-model upon a particular fixing of the “gauge freedom”; (iii) the corresponding (ρ, τ)-geometry of the manifold of ϕ-exponential family can have different appearances (gauge freedom), such as a Hessian geometry (under one type of gauge selection) and a conformal Hessian geometry (under another type of gauge selection). The work of Naudts and Zhang (2018) unifies intermediary results in Ohara et al. (2012), Amari et al. (2012), and Matsuzoe (2014), and provides a general deformation framework that preserves the rigid interlock of (i) the function form of entropy, cross-entropy, and relative entropy (divergence); (ii) the functional form of the probability family with the corresponding normalization and potential, and the duality between the natural and expectation parameterizations; (iii) the expressions of the Riemannian metric (Fisher–Rao metric in general and Hessian metric in particular) and of the conjugate connections. The deformation approaches described above are based on the application of Legendre (convex) duality to various embedding functions.

    • Introduction to nonextensive statistical mechanics: Approaching a complex world: Second Edition

      2023, Introduction to Nonextensive Statistical Mechanics: Approaching a Complex World: Second Edition
    View all citing articles on Scopus
    View full text