Abstract
In 1979, J.M. Bernardo argued heuristically that in the case of regular product experiments his information theoretic reference prior is equal to Jeffreys' prior. In this context, B.S. Clarke and A.R. Barron showed in 1994, that in the same class of experiments Jeffreys' prior is asymptotically optimal in the sense of Shannon, or, in Bayesian terms, Jeffreys' prior is asymptotically least favorable under Kullback Leibler risk. In the present paper, we prove, based on Clarke and Barron's results, that every sequence of Shannon optimal priors on a sequence of regular iid product experiments converges weakly to Jeffreys' prior. This means that for increasing sample size Kullback Leibler least favorable priors tend to Jeffreys' prior.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arimoto, S. (1972). An algorithm for computing the capacity of arbitrary discrete memoryless channels.IEEE Transactions on Information Theory,18, 14–20.
Blahut, R.E. (1972). Computation of channel capacity and rate distortion functions.IEEE Transactions on Information Theory,18, 460–473.
Berger, J. and J.M. Bernardo (1992). On the development of the reference prior method.Bayesian Statistics 4 (J.M. Bernardo, J.O. Berger, A.P. Dawid, A.F.M. Smith, eds.) Oxford: University Press, 35–60 (with discussion).
Berger, J., J.M. Bernardo and M. Mendoza (1989). On priors that maximize expected information.Recent Developments in Statistics and Their Applications (J.P. Klein and J.C. Lee, eds.). Seoul: Freedom Academy Publishing. 1–20.
Bernardo, J.M. (1979). Reference posterior distributions for Bayesian inference.Journal of the Royal Statistical Society, Ser. B,41, 113–147 (with discussion).
Bernardo, J.M. and A.F.M. Smith (1994).Bayesian theory. Chichester: Wiley and Sons.
Clarke, B.S. and A.R. Barron (1990). Information—theoretic asymptotics of Bayes methods.IEEE Transactions on Information Theory,36, 453–471.
Clarke, B.S. and A.R. Barron (1994). Jeffreys' prior is asymptotically least favorable under entropy risk.Journal of Statistical Plannning and Inference,41, 37–60.
Dudley, R.M. (1989).Real analysis and probability. Pacific Grove, California: Wadsworth and Brooks/Cole.
Gallager, R.G. (1968).Information theory and reliable communication. New York: Wiley and Sons.
Ghosal, S. (1997). Reference priors in multiparameter nonregular cases.Test,6, 159–186.
Gosh, J.K. and R. Mukerjee. (1992). Non-informative priors.Bayesian Statistics 4 (J.M. Bernardo, J.O. Berger, A.P. Dawid, A.F.M. Smith, eds.). Oxford: University Press, 195–210 (with discussion).
Haussler, D. (1997). A general minimax result for relative entropy.IEEE Transactions on Information Theory,43, 1276–1280.
Jeffreys, H. (1961/1983).Theory of probability (3rd. ed.). Oxford: Clarendon Press.
Krob, J. (1992).Kapazität statistischer Experimente. Ph.D. Dissertation. Kaiserslautern: Department of Mathematics, University of Kaiserslautern.
Krob, J. and H. Scholl (1997). A minimax result for the Kullback Leibler Bayes risk.Economic quality control, to appear.
Lindley, D.V. (1956). On a measure of the information provided by an experiment.Annals of Mathematical Statatistics,27, 986–1005.
Meyer, P.-A. (1966).Probabilités et potential. Paris: Maison d'edition Hermann.
Shannon, C.E. and W. Weaver (1949).The mathematical theory of communication. Urbana: University of Illinois Press.
Spall, J.C. and S.D. Hill (1990). Least informative Bayesian prior distributions for finite samples based on information theory.IEEE Transactions on Automatic Control,35, 580–583.
Sun, D. and K.Y. Ye (1995). Reference prior Bayesian analysis for normal mean products.Journal of the American Statistical Association,90, 589–597.
Topsøe, F. (1974).Informationstheorie. Stuttgart: B.G. Teubner Verlag.
Williams, D. (1991).Probability with martingales. Cambridge: University Press.
Yang, R. (1995). Invariance of the reference prior under reparametrization.Test,4, 83–94.
Yang, R. and J.O. Berger (1994). Estimation of a covariance matrix using the reference prior.Annals of Statistics,22, 1195–2111.
Zhang, Z. (1994).Discrete noninformative priors. Ph.D. Dissertation. New Haven: Department of Statistics, Yale University.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Scholl, H.R. Shannon optimal priors on independent identically distributed statistical experiments converge weakly to Jeffreys' prior. Test 7, 75–94 (1998). https://doi.org/10.1007/BF02565103
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02565103