Abstract
The adaptive Metropolis (AM) algorithm of Haario, Saksman and Tamminen (Bernoulli 7(2):223–242, 2001) uses the estimated covariance of the target distribution in the proposal distribution. This paper introduces a new robust adaptive Metropolis algorithm estimating the shape of the target distribution and simultaneously coercing the acceptance rate. The adaptation rule is computationally simple adding no extra cost compared with the AM algorithm. The adaptation strategy can be seen as a multidimensional extension of the previously proposed method adapting the scale of the proposal distribution in order to attain a given acceptance rate. The empirical results show promising behaviour of the new algorithm in an example with Student target distribution having no finite second moment, where the AM covariance estimate is unstable. In the examples with finite second moments, the performance of the new approach seems to be competitive with the AM algorithm combined with scale adaptation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrieu, C., Moulines, É.: On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Probab. 16(3), 1462–1505 (2006)
Andrieu, C., Robert, C.P.: Controlled MCMC for optimal sampling. Tech. Rep. Ceremade 0125, Université Paris Dauphine (2001)
Andrieu, C., Thoms, J.: A tutorial on adaptive MCMC. Stat. Comput. 18(4), 343–373 (2008)
Andrieu, C., Moulines, É., Volkov, S.: Convergence of stochastic approximation for Lyapunov stable dynamics: a proof from first principles. Technical report (2004)
Andrieu, C., Moulines, É., Priouret, P.: Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44(1), 283–312 (2005)
Atchadé, Y., Fort, G.: Limit theorems for some adaptive MCMC algorithms with subgeometric kernels. Bernoulli 16(1), 116–154 (2010)
Atchadé, Y.F., Rosenthal, J.S.: On adaptive Markov chain Monte Carlo algorithms. Bernoulli 11(5), 815–828 (2005)
Benveniste, A., Métivier, M., Priouret, P.: Adaptive Algorithms and Stochastic Approximations. Applications of Mathematics, vol. 22. Springer, Berlin (1990)
Borkar, V.S.: Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press, Cambridge (2008)
Dongarra, J.J., Bunch, J.R., Moler, C.B., Stewart, G.W.: LINPACK Users’ Guide. Society for Industrial and Applied Mathematics (1979)
Gelman, A., Roberts, G.O., Gilks, W.R.: Efficient Metropolis jumping rules. In: Bayesian Statistics 5, pp. 599–607. Oxford University Press, Oxford (1996)
Gilks, W.R., Richardson, S., Spiegelhalter, D.J.: Markov Chain Monte Carlo in Practice. Chapman & Hall/CRC, Boca Raton (1998)
Haario, H., Saksman, E., Tamminen, J.: An adaptive Metropolis algorithm. Bernoulli 7(2), 223–242 (2001)
Hastie, D.: Toward automatic reversible jump Markov chain Monte Carlo. PhD thesis, University of Bristol (2005)
Huber, P.J.: Robust Statistics. Wiley Series in Probability and Mathematical Statistics. Wiley, New York (1981)
Jarner, S.F., Hansen, E.: Geometric ergodicity of Metropolis algorithms. Stoch. Process. Appl. 85, 341–361 (2000)
Jarner, S.F., Roberts, G.O.: Convergence of heavy-tailed Monte Carlo Markov chain algorithms. Scand. J. Stat. 34(4), 781–815 (2007)
Kushner, H.J., Yin, G.G.: Stochastic Approximation and Recursive Algorithms and Applications, 2nd edn. Applications of Mathematics: Stochastic Modelling and Applied Probability, vol. 35. Springer, Berlin (2003)
Nummelin, E.: MC’s for MCMC’ists. Int. Stat. Rev. 70(2), 215–240 (2002)
Robert, C.P., Casella, G.: Monte Carlo Statistical Methods. Springer, New York (1999)
Roberts, G.O., Rosenthal, J.S.: Optimal scaling for various Metropolis-Hastings algorithms. Stat. Sci. 16(4), 351–367 (2001)
Roberts, G.O., Rosenthal, J.S.: General state space Markov chains and MCMC algorithms. Probab. Surv. 1, 20–71 (2004)
Roberts, G.O., Rosenthal, J.S.: Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. J. Appl. Probab. 44(2), 458–475 (2007)
Roberts, G.O., Rosenthal, J.S.: Examples of adaptive MCMC. J. Comput. Graph. Stat. 18(2), 349–367 (2009)
Roberts, G.O., Gelman, A., Gilks, W.R.: Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7(1), 110–120 (1997)
Saksman, E., Vihola, M.: On the ergodicity of the adaptive Metropolis algorithm on unbounded domains. Ann. Appl. Probab. 20(6), 2178–2203 (2010)
Vihola, M.: Grapham: Graphical models with adaptive random walk Metropolis algorithms. Comput. Stat. Data Anal. 54(1), 49–54 (2010)
Vihola, M.: Can the adaptive Metropolis algorithm collapse without the covariance lower bound? Electron. J. Probab. 16, 45–75 (2011a)
Vihola, M.: On the stability and ergodicity of adaptive scaling Metropolis algorithms. Preprint (2011b). arXiv:0903.4061v3
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Vihola, M. Robust adaptive Metropolis algorithm with coerced acceptance rate. Stat Comput 22, 997–1008 (2012). https://doi.org/10.1007/s11222-011-9269-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-011-9269-5