Physica A: Statistical Mechanics and its Applications
Geometry of deformed exponential families: Invariant, dually-flat and conformal geometries
Highlights
► Information geometries on a manifold of a deformed exponential family are studied. ► They define respective quantities such as deformed free energies and entropies. ► Geometry of the -exponential family is characterized via invariance and flatness. ► Conformal structure of the flat geometry of the -exponential family is studied.
Introduction
Since the introduction of -entropy by Tsallis [1] (see also an extensive monograph [2]), much attention has been paid to non-extensive statistical mechanics. It is related to various ‘non-standard phenomena’ subject to the power law not only in statistical physics but in economics and disaster statistics. Here, families of probability distributions of the -exponential family and more general deformed exponential families play a major role. In the present paper, a geometrical foundation is given to these families of distributions from the point of view of information geometry [3].
The deformed exponential family was introduced and studied extensively by Naudts [4], [5] (see also a monograph [6]). Kaniadakis et al. [7] studied the -exponential family which belongs to the deformed exponential family. Its mathematical structure was studied by Pistone [8] and Vigelis and Cavalcante [9]. See other examples with interesting discussions [10], [11]. In statistics, a similar notion of a generalized exponential family [12] or the U-model [13], [14] is discussed on the bases of respective motives.
Many useful concepts such as generalized entropy, divergence and escort probability distribution have been proposed. However, their relationships have not necessarily been well understood theoretically and are waiting for further geometrical and statistical elucidation. It is also useful to characterize the -families in the class of general deformed exponential families.
In the present study, information geometry [3] is used to give a foundation to the deformed exponential families. Two types of geometry can be introduced in the manifold of a deformed exponential family: One is the invariant geometry, where the Fisher information is the unique Riemannian metric (Chentsov [15]; also see Ref. [3]) together with a dual pair of invariant affine connections (-connections). The other is the dually flat geometry [3] (also see Ref. [16]), which is not necessarily invariant but accompanies the Legendre structure. The escort probability distribution belongs to the latter geometry. The two geometries give different free-energies, entropies and divergences in general.
The exponential and mixture families are characterized by the property that they sit at the intersection of the classes of the invariant and flat geometries. The -exponential family is then characterized from two viewpoints: One is invariance and flatness in the class of positive measures and the other is conformal geometry [17], [18]. It is shown that the -family is a unique class of flat geometry that is connected conformally to the invariant geometry.
Section snippets
Deformed exponential family
We follow Naudts [6], [19] for the formulation of the deformed exponential family. Given a positive increasing function on , a deformed logarithm, called the -logarithm, is defined by This is a concave monotonically increasing function. When is a power function, (1) gives the -logarithm The ordinary logarithm is obtained by taking the limit of .
The inverse of the -logarithm is the -exponential, given by
-geometry
A -family (7) is considered as a manifold with a (local) coordinate system . We use the invariance principle to introduce a unique geometrical structure (see Refs. [3], [15]).
Invariance principle: The geometry is invariant under the transformation of random variable to , provided is a sufficient statistics.
The invariant geometry is characterized by the two tensors, where and denotes expectation. The
Flat -geometry
Apart from the invariancy principle, we give a new dually flat structure to the -family, different from the invariant geometry. This is called the -geometry. When , we particularly call it the -geometry. The -free-energy of a -family is known to be a convex function [6]. We give its simple proof for later use. We have, by putting for simplicity, Since
Conformal transformation connecting invariant and flat geometries
Two geometries, invariant and flat, induced in the -structure of were studied in the previous sections. It is interesting to know how they are related. In order to answer this question, we calculate the -Fisher metric by using the -divergence.
Theorem 12 The -Fisher metric of is given by
Proof The Taylor expansion of the -divergence is calculated as Hence, the -Fisher
Conclusions
The geometrical structures of the manifold of a general deformed exponential family were studied. The two geometrical structures were introduced, one from the viewpoint of invariance and the other from the viewpoint of flatness. They give different definitions of generalized free energy, entropy and divergence. The exponential and mixture families are characterized uniquely by the fact that the two geometries coincide. The -exponential families are characterized by the fact that the two
References (32)
- et al.
Deformed logarithms and entropies
Physica A
(2004) Entropy and equilibrium via games of complexity
Physica A
(2004)Deformed exponentials and logarithms in generalized thermostatistics
Physica A
(2002)Geometry of distributions associated with Tsallis statistics and properties of relative entropy minimization
Phys. Lett. A
(2007)Possible generalization of Boltzmann–Gibbs statistics
J. Stat. Phys.
(1988)Introduction to Nonextensive Statistical Mechanics
(2009)- et al.
Methods of Information Geometry (Translations of Mathematical Monographs)
(2000) Estimators, escort probabilities, and phi-exponential families in statistical physics
J. Ineq. Pure Appl. Math.
(2004)Generalized exponential families and associated entropy functions
Entropy
(2008)Generalized Thermostatistics
(2011)
-exponential models from the geometrical viewpoint
Eur. Phys. J. B
On the -exponential family of probability distributions
J. Theor. Probab.
Generalized information and entropy measures in physics
Contemp. Phys.
Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
Ann. Stat.
Information geometry and statistical pattern recognition
Sugaku Expositions
A generalization of independence in naive Bayes model
Lect. Notes Comput. Sci.
Cited by (71)
Informative fractal dimension associated with nonmetricity in information geometry
2023, Physica A: Statistical Mechanics and its ApplicationsIndependent Approximates enable closed-form estimation of heavy-tailed distributions
2022, Physica A: Statistical Mechanics and its Applicationsλ-Deformed probability families with subtractive and divisive normalizations
2021, Handbook of StatisticsCitation Excerpt :It eventually became clear in the 2018 paper Naudts and Zhang (2018) by Naudts and the first author that (i) the ϕ-model and U-model turned out to be equivalent; (ii) they are special cases of the (ρ, τ)-model upon a particular fixing of the “gauge freedom”; (iii) the corresponding (ρ, τ)-geometry of the manifold of ϕ-exponential family can have different appearances (gauge freedom), such as a Hessian geometry (under one type of gauge selection) and a conformal Hessian geometry (under another type of gauge selection). The work of Naudts and Zhang (2018) unifies intermediary results in Ohara et al. (2012), Amari et al. (2012), and Matsuzoe (2014), and provides a general deformation framework that preserves the rigid interlock of (i) the function form of entropy, cross-entropy, and relative entropy (divergence); (ii) the functional form of the probability family with the corresponding normalization and potential, and the duality between the natural and expectation parameterizations; (iii) the expressions of the Riemannian metric (Fisher–Rao metric in general and Hessian metric in particular) and of the conjugate connections. The deformation approaches described above are based on the application of Legendre (convex) duality to various embedding functions.
Introduction to nonextensive statistical mechanics: Approaching a complex world: Second Edition
2023, Introduction to Nonextensive Statistical Mechanics: Approaching a Complex World: Second Edition