Skip to main content
SpringerLink
Log in

Summary statistics and discrepancy measures for approximate Bayesian computation via surrogate posteriors

  • OriginalPaper
  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

A key ingredient in approximate Bayesian computation (ABC) procedures is the choice of a discrepancy that describes how different the simulated and observed data are, often based on a set of summary statistics when the data cannot be compared directly. Unless discrepancies and summaries are available from experts or prior knowledge, which seldom occurs, they have to be chosen, and thus their choice can affect the quality of approximations. The choice between discrepancies is an active research topic, which has mainly considered data discrepancies requiring samples of observations or distances between summary statistics. In this work, we introduce a preliminary learning step in which surrogate posteriors are built from finite Gaussian mixtures using an inverse regression approach. These surrogate posteriors are then used in place of summary statistics and compared using metrics between distributions in place of data discrepancies. Two such metrics are investigated: a standard L\(_2\) distance and an optimal transport-based distance. The whole procedure can be seen as an extension of the semi-automatic ABC framework to a functional summary statistics setting and can also be used as an alternative to sample-based approaches. The resulting ABC quasi-posterior distribution is shown to converge to the true one, under standard conditions. Performance is illustrated on both synthetic and real data sets, where it is shown that our approach is particularly useful when the posterior is multimodal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Akesson, M., Singh, P., Wrede, F., Hellander, A.: Convolutional neural networks as summary statistics for approximate Bayesian computation. IEEE/ACM Trans. Comput. Biol. Bioinformat. (2021)

  • An, Z., Nott, D.J., Drovandi, C.: Robust Bayesian synthetic likelihood via a semi-parametric approach. Stat. Comput. 30(3), 543–557 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • An, Z., South, L.F., Nott, D.J., Drovandi, C.C.: Accelerating Bayesian synthetic likelihood with the graphical lasso. J. Comput. Gr. Stat. 28(2), 471–475 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Arridge, S., Maass, P., Öktem, O., Schönlieb, C.-B.: Solving inverse problems using data-driven models. Acta Numer 28, 1–174 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Bernard-Michel, C., Douté, S., Fauvel, M., Gardes, L., Girard, S.: Retrieval of Mars surface physical properties from OMEGA hyperspectral images using Regularized Sliced Inverse Regression. J. Geophys. Res. Planets, 114(E6) (2009)

  • Bernton, E., Jacob, P.E., Gerber, M., Robert, C.P.: Approximate Bayesian computation with the Wasserstein distance. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 81, 235–269 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Bishop, C.M.: Mixture density networks. Technical report, Aston University, Birmingham (1994)

  • Blum, M.G.B., Nunes, M.A., Prangle, D., Sisson, S.A.: A comparative review of dimension reduction methods in approximate Bayesian computation. Stat. Sci. 28(2), 189–208 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Boux, F., Forbes, F., Arbel, J., Lemasson, B., Barbier, E.L.: Bayesian inverse regression for vascular magnetic resonance fingerprinting. IEEE Trans. Med. Imaging 40(7), 1827–1837 (2021)

    Article  Google Scholar 

  • Buchholz, A., Chopin, N.: Improving approximate Bayesian computation via quasi-monte Carlo. J. Comput. Graph. Stat. 28(1), 205–219 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Cappé, O., Moulines, E.: On-line expectation-maximization algorithm for latent data models. J. R. Stat. Soc. B 71, 593–613 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Chen, Y., Georgiou, T.T., Tannenbaum, A.: Optimal transport for gaussian mixture models. IEEE Access 7, 6269–6278 (2019)

    Article  Google Scholar 

  • Chen, Y., Zhang, D., Gutmann, M., Courville, A., Zhu, Z.: Neural approximate sufficient statistics for implicit models. In: ICLR2021 spotlight (2021)

  • Cook, R.D., Forzani, L.: Partial least squares prediction in high-dimensional regression. Ann. Stat. 47(2), 884–908 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Crackel, R., Flegal, J.: Bayesian inference for a flexible class of bivariate beta distributions. J. Stat. Comput. Simul. 87, 295–312 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Csillery, K., Francois, O., Blum, M.: abc: an R package for approximate Bayesian computation (ABC). Methods Ecol. Evol. (2012)

  • Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential monte Carlo method for approximate Bayesian computation. Stat. Comput. 22(5), 1009–1020 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Deleforge, A., Forbes, F., Ba, S., Horaud, R.: Hyper-spectral image analysis with partially-latent regression and spatial Markov dependencies. IEEE J. Sel. Top. Signal Process. 9(6), 1037–1048 (2015)

    Article  Google Scholar 

  • Deleforge, A., Forbes, F., Horaud, R.: High-dimensional regression with gaussian mixtures and partially-latent response variables. Stat. Comput. 25(5), 893–911 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Delon, J., Desolneux, A.: A Wasserstein-type distance in the space of Gaussian Mixture Models. SIAM J. Imaging Sci. (2020)

  • Dinh, L., Krueger, D., Bengio, Y.: NICE: non-linear independent components estimation. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015. Workshop Track Proceedings (2015)

    Google Scholar 

  • Drovandi, C., Pettitt, T., Lee, A.: Bayesian indirect inference using a parametric auxiliary model. Stat. Sci. 30(1), 72–95 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Drovandi, C.C., Pettitt, A.N.: Likelihood-free Bayesian estimation of multivariate quantile distributions. Comput. Stat. Data Anal. 55, 2541–2556 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Fearnhead, P., Prangle, D.: Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 74(3), 419–474 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Fernando, J., Schmidt, F., Douté, S.: Martian surface microtexture from orbital CRISM multi-angular observations: A new perspective for the characterization of the geological processes. Planet. Space Sci. 128, 30–51 (2016)

    Article  Google Scholar 

  • Frazier, D.T., Drovandi, C.: Robust approximate Bayesian inference with synthetic likelihood. J. Comput. Gr. Stat. 1–19 (2021)

  • Greenberg, D., Nonnenmacher, M., Macke, J.: Automatic posterior transformation for likelihood-free inference. In: International Conference on Machine Learning, pp. 2404–2414. PMLR (2019)

  • Gutmann, M.U., Dutta, R., Kaski, S., Corander, J.: Likelihood-free inference via classification. Stat. Comput. 28, 411–425 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  • Hovorka, R., Canonico, V., Chassin, L.J., Haueter, U., Massi-Benedetti, M., Federici, M.O., Pieber, T.R., Schaller, H.C., Schaupp, L., Vering, T., Wilinska, M.E.: Nonlinear model predictive control of glucose concentration in subjects with type 1 diabetes. Physiol. Meas. 25(4), 905–920 (2004)

    Article  Google Scholar 

  • Ingrassia, S., Minotti, S.C., Vittadini, G.: Local statistical modeling via a cluster-weighted approach with elliptical distributions. J. Classif. 29(3), 363–401 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Jacob, P., Bernton, E., Gerber, M., Robert, C.P.: Winference: R package to perform approximate Bayesian computation with the Wasserstein distance (2020)

  • Jiang, B., Wu, T.-Y., C., Z., Wong, W.: Learning summary statistics for approximate Bayesian computation via deep neural network. Stat. Sinica, pp. 1595–1618 (2017)

  • Jiang, B., Wu, T.-Y., Wong, W.H.: Approximate Bayesian computation with Kullback–Leibler divergence as data discrepancy. In: 21st International Conference on Artificial Intelligence and Statistics (AISTATS) (2018)

  • Kobyzev, I., Prince, S., Brubaker, M.: Normalizing flows: an introduction and review of current methods. IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1 (2020)

  • Kristan, M., Leonardis, A., Skočaj, D.: Multivariate online kernel density estimation with Gaussian kernels. Pattern Recogn. 44(10–11), 2630–2642 (2011)

    Article  MATH  Google Scholar 

  • Kruse, J., Ardizzone, L., Rother, C., Kothe, U.: Benchmarking invertible architectures on inverse problems. In: Workshop on Invertible Neural Networks and Normalizing Flows (ICML 2019), arXiv preprint arXiv:2101.10763 (2021)

  • Kugler, B., Forbes, F., Douté, S.: Fast Bayesian inversion for high dimensional inverse problems. To appear in Statistics and Computing, https://hal.archives-ouvertes.fr/hal-02908364 (2021)

  • Labarre, S.: Caractérisation et modélisation de la rugosité multi-échelle des surfaces naturelles par télédétection dans le domaine solaire. PhD thesis, Physique Univers Sorbonne Paris Cité. Supervised by C. Ferrari and S. Jacquemoud (2017)

  • Lemasson, B., Pannetier, N., Coquery, N., Boisserand, L.S.B., Collomb, N., Schuff, N., Moseley, M., Zaharchuk, G., Barbier, E.L., Christen, T.: MR vascular fingerprinting in stroke and brain tumors models. Sci. Rep. 6, 37071 (2016)

    Article  Google Scholar 

  • Li, K.-C.: Sliced inverse regression for dimension reduction. J. Am. Stat. Assoc. 86(414), 316–327 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  • Lueckmann, J.-M., Boelts, J., Greenberg, D.S., Gonçalves, P.J., Macke, J.H.: Benchmarking simulation-based inference. In: Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 130 of Proceedings of Machine Learning Research, pp. 343–351. PMLR (2021)

  • Lueckmann, J.-M., Goncalves, P.J., Bassetto, G., Öcal, K., Nonnenmacher, M., Macke, J.H.: Flexible statistical inference for mechanistic models of neural dynamics. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Adv. Neural Inf. Process. Syst., vol. 30. Curran Associates Inc, Red Hook, NY (2017)

    Google Scholar 

  • Ma, D., Gulani, V., Seiberlich, N., Liu, K., Sunshine, J.L., Duerk, J.L., Griswold, M.A.: Magnetic resonance fingerprinting. Nature 495(7440), 187–192 (2013)

    Article  Google Scholar 

  • Marin, J.-M., Pudlo, P., Robert, C.P., Ryder, R.J.: Approximate Bayesian computation methods. Stat. Comput. 22, 1167–1180 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Mesejo, P., Saillet, S., David, O., Bénar, C., Warnking, J.M., Forbes, F.: A differential evolution-based approach for fitting a nonlinear biophysical model to fMRI BOLD data. IEEE J. Sel. Top. Signal Process. 10(2), 416–427 (2016)

    Article  Google Scholar 

  • Muandet, K., Fukumizu, K., Dinuzzo, F., Scholkopf, B.: Learning from distributions via support measure machines. In: Advances in Neural Information Processing Systems, pp. 10–18 (2012)

  • Murchie, S.L., Seelos, F.P., Hash, C.D., Humm, D.C., Malaret, E., McGovern, J.A., Choo, T.H., Seelos, K.D., Buczkowski, D.L., Morgan, M.F., Barnouin-Jha, O.S., Nair, H., Taylor, H.W., Patterson, G.W., Harvel, C.A., Mustard, J.F., Arvidson, R.E., McGuire, P., Smith, M.D., Wolff, M.J., Titus, T.N., Bibring, J.-P., Poulet, F.: Compact reconnaissance imaging spectrometer for mars investigation and data set from the mars reconnaissance orbiter’s primary science phase. J. Geophys. Res Planets, 114(E2):E00D07 (2009)

  • Nataraj, G., Nielsen, J.-F., Scott, C., Fessler, J.A.: Dictionary-free MRI PERK: parameter estimation via regression with kernels. IEEE Trans. Med. Imaging 37(9), 2103–2114 (2018)

    Article  Google Scholar 

  • Nguyen, H., Forbes, F.: Global implicit function theorems and the online expectation–maximisation algorithm. Austral. N. Z. J. Stat., to appear (2022)

  • Nguyen, H.D., Arbel, J., Lu, H., Forbes, F.: Approximate Bayesian computation via the energy statistic. IEEE Access 8, 131683–131698 (2020)

    Article  Google Scholar 

  • Nguyen, H.D., Chamroukhi, F., Forbes, F.: Approximation results regarding the multiple-output Gaussian gated mixture of linear experts model. Neurocomputing (2019)

  • Nguyen, H.D., Forbes, F., McLachlan, G.: Mini-batch learning of exponential family finite mixture models. Stat. Comput. 30, 731–748 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • Nguyen, H.D., Nguyen, T., Chamroukhi, F., McLachlan, G.J.: Approximations of conditional probability density functions in Lebesgue spaces via mixture of experts models. J. Stat. Distrib. Appl. 8(1), 13 (2021)

    Article  MATH  Google Scholar 

  • Nguyen, T., Chamroukhi, F., Nguyen, H.D., McLachlan, G.J.: Approximation of probability density functions via location-scale finite mixtures in Lebesgue spaces. Commun. Stat. Theor. Methods 1–12 (2022)

  • Nguyen, T., Nguyen, H.D., Chamroukhi, F., Forbes, F.: A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models. Electron. J. Stat. (2022) (to appear)

  • Nguyen, T., Chamroukhi, F., Nguyen, H.D., McLachlan, G.J.: Approximation of probability density functions via location-scale finite mixtures in Lebesgue spaces. arXiv preprint arXiv:2008.09787. To appear. Communications in Statistics - Theory and Methods (2020c)

  • Nguyen, T., Nguyen, H.D., Chamroukhi, F., Forbes, F.: A non-asymptotic penalization criterion for model selection in mixture of experts models. To appear in Electronic Journal of Statistics (2021b)

  • Nguyen, T., Nguyen, H.D., Chamroukhi, F., McLachlan, G.J.: Approximation by finite mixtures of continuous density functions that vanish at infinity. Cogent Math. Stat. 7(1), 1750861 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • Nunes, M.A., Prangle, D.: abctools: An R package for tuning approximate bayesian computation analyses. https://cran.r-project.org/web/packages/abctools/ (2015)

  • Ong, V., Nott, D., Tran, M.-N., Sisson, S., Drovandi, C.: Likelihood-free inference in high dimensions with synthetic likelihood. Comput. Stat. Data Anal. 128 (2018)

  • Papamakarios, G., Murray, I.: Fast \(\varepsilon \)-free inference of simulation models with Bayesian conditional density estimation. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates Inc (2016)

    Google Scholar 

  • Park, M., Jitkrittum, W., Sejdinovic, D.: K2-ABC: approximate Bayesian computation with kernel embeddings. In: 19th International Conference on Artificial Intelligence and Statistics (AISTATS) (2016)

  • Perthame, E., Forbes, F., Deleforge, A., Devijver, E., Gallopin, M.: xLLiM: high dimensional locally-linear mapping. R Pack. Vers. 2, 1 (2017)

    Google Scholar 

  • Pilorget, C., Fernando, J., Ehlmann, B.L., Schmidt, F., Hiroi, T.: Wavelength dependence of scattering properties in the VIS–NIR and links with grain-scale physical and compositional properties. Icarus 267, 296–314 (2016)

    Article  Google Scholar 

  • Prangle, D.: Adapting the ABC distance function. Bayesian Anal. 12(1), 289–309 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Prangle, D., Everitt, R.G., Kypraios, T.: A rare event approach to high-dimensional approximate Bayesian computation. Stat. Comput. 28, 819–834 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  • Price, L.F., Drovandi, C.C., Lee, A., Nott, D.J.: Bayesian synthetic likelihood. J. Comput. Graph. Stat. 27(1), 1–11 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  • Rakhlin, A., Panchenko, D., Mukherjee, S.: Risk bounds for mixture density estimation. ESAIM Probab. Stat. 9, 220–229 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Rodrigues, G.S., Nott, D.J., Sisson, S.A.: Functional regression approximate Bayesian computation for Gaussian process density estimation. Comput. Stat. Data Anal. 103, 229–241 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Rubio, F., Johansen, A.M.: A simple approach to maximum intractable likelihood estimation. Electron. J. Stat. 7, 1632–1654 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Schmidt, F., Fernando, J.: Realistic uncertainties on Hapke model parameters from photometric measurements. Icarus 260, 73–93 (2015)

    Article  Google Scholar 

  • Sisson, S.A., Fan, Y., Beaumont, M.A. (eds.): Handbook of Approximate Bayesian Computation. CRC Press, Boca Raton (2019)

    MATH  Google Scholar 

  • Soubeyrand, S., Carpentier, F., Guiton, F., Klein, E.K.: Approximate Bayesian computation with functional statistics. Stat. Appl. Genet. Mol. Biol. 12(1), 17–37 (2013)

    Article  MathSciNet  Google Scholar 

  • Sriperumbudur, B.K., Gretton, A., Fukumizu, K., Scholkopf, B., Lanckriet, G.R.: Hilbert space embeddings and metrics on probability measures. J. Mach. Learn. Res. 11, 1517–1561 (2010)

  • Wang, F., Syeda-Mahmood, T., Vemuri, B. C., Beymer, D., Rangarajan, A.: Closed-form Jensen-Renyi divergence for mixture of Gaussians and applications to group-wise shape registration. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 648–655. Springer (2009)

  • Wiqvist, S., Mattei, P.-A., Picchini, U., Frellsen, J.: Partially exchangeable networks and architectures for learning summary statistics in approximate Bayesian computation. In: Chaudhuri, K., Salakhutdinov, R. (eds) Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 6798–6807, Long Beach, California, USA (2019)

  • Wood, S.: Statistical inference for noisy nonlinear ecological dynamic systems. Nature 466(7310), 1102–1104 (2010)

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to reviewers and editors for their time and comments on this work, which have helped us in producing a much-improved manuscript. FF would like to thank Guillaume Kon Kam King for an initial discussion on semi-automatic ABC, which inspired this work, Benoît Kugler and Sylvain Douté for providing the simulations for the planetary science example, and for helpful discussions on the Hapke model. The work is supported by Inria project LANDER.

Author information

Authors and Affiliations

  1. University of Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Inria Grenoble Rhone-Alpes, 655 av. de l’Europe, 38335, Montbonnot, France

    Florence Forbes, TrungTin Nguyen & Julyan Arbel

  2. School of Mathematics and Physics, University of Queensland, St. Lucia, QLD, Australia

    Hien Duy Nguyen

  3. Normandie University, UNICAEN, CNRS, LMNO, Caen, 14000, France

    TrungTin Nguyen

Authors
  1. Florence Forbes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hien Duy Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. TrungTin Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Julyan Arbel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florence Forbes.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 2172 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Forbes, F., Nguyen, H.D., Nguyen, T. et al. Summary statistics and discrepancy measures for approximate Bayesian computation via surrogate posteriors. Stat Comput 32, 85 (2022). https://doi.org/10.1007/s11222-022-10155-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11222-022-10155-6

Keywords

Search

Navigation