Abstract
We present a novel computational approach for extracting localized signals from smooth background distributions. We focus on datasets that can be naturally presented as binned integer counts, demonstrating our procedure on the CERN open dataset with the Higgs boson signature, from the ATLAS collaboration at the Large Hadron Collider. Our approach is based on Gaussian Process (GP) regression — a powerful and flexible machine learning technique which has allowed us to model the background without specifying its functional form explicitly and separately measure the background and signal contributions in a robust and reproducible manner. Unlike functional fits, our GP-regression-based approach does not need to be constantly updated as more data becomes available. We discuss how to select the GP kernel type, considering trade-offs between kernel complexity and its ability to capture the features of the background distribution. We show that our GP framework can be used to detect the Higgs boson resonance in the data with more statistical significance than a polynomial fit specifically tailored to the dataset. Finally, we use Markov Chain Monte Carlo (MCMC) sampling to confirm the statistical significance of the extracted Higgs signature.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
D.S. Sivia and W.I.F. David, A Bayesian approach to extracting structure-factor amplitudes from powder diffraction data, Acta Crystallographica A 50 (1994) 703.
W.I.F. David and D.S. Sivia, Background estimation using a robust Bayesian analysis, J. Appl. Crystallography 34 (2001) 318.
T.A. Gordon, E. Agol and D. Foreman-Mackey, A fast, two-dimensional Gaussian process method based on celerite: applications to transiting exoplanet discovery and characterization, Astron. J. 160 (2020) 240.
D. Foreman-Mackey, E. Agol, S. Ambikasaran and R. Angus, Fast and scalable Gaussian process modeling with applications to astronomical time series, Astron. J. 154 (2017) 220.
ATLAS collaboration, Observation of a new particle in the search for the standard model Higgs boson with the ATLAS detector at the LHC, Phys. Lett. B 716 (2012) 1 [arXiv:1207.7214] [INSPIRE].
Particle Data Group collaboration, Review of particle physics, PTEP 2020 (2020) 083C01 [INSPIRE].
C. Bishop, Pattern recognition and machine learning, Springer (2006).
P. Mehta et al., A high-bias, low-variance introduction to machine learning for physicists, Phys. Rept. 810 (2019) 1 [arXiv:1803.08823] [INSPIRE].
J.W. Rocks and P. Mehta, Memorizing without overfitting: bias, variance, and interpolation in over-parameterized models, arXiv:2010.13933.
S.S. Wilks, The large-sample distribution of the likelihood ratio for testing composite hypotheses, Annals Math. Statist. 9 (1938) 60 [INSPIRE].
G. Cowan, K. Cranmer, E. Gross and O. Vitells, Asymptotic formulae for likelihood-based tests of new physics, Eur. Phys. J. C 71 (2011) 1554 [Erratum ibid. 73 (2013) 2501] [arXiv:1007.1727] [INSPIRE].
ATLAS collaboration, Search for new phenomena in dijet mass and angular distributions from pp collisions at \( \sqrt{s} \) = 13 TeV with the ATLAS detector, Phys. Lett. B 754 (2016) 302 [arXiv:1512.01530] [INSPIRE].
M. Titsias, Variational learning of inducing variables in sparse Gaussian processes, in Proceedings of the twelth international conference on artificial intelligence and statistics, D. van Dyk and M. Welling eds., Proc. Machine Learning Research 5, Hilton Clearwater Beach Resort, Clearwater Beach, FL, U.S.A., 16–18 April 2009, p. 567.
C.E. Rasmussen and C.K.I. Williams, Gaussian processes for machine learning, The MIT Press (2005).
K. Kersting, C. Plagemann, P. Pfaff and W. Burgard, Most likely heteroscedastic Gaussian process regression, in Proceedings of the 24th international conference on machine learning, ACM (2007).
A. O’Hagan, Curve fitting and optimal design for prediction, J. Roy. Statist. Soc. B 40 (1978) 1.
K.G. Iyer et al., Nonparametric star formation history reconstruction with Gaussian processes. I. Counting major episodes of star formation, Astrophys. J. 879 (2019) 116.
C.J. Moore, C.P.L. Berry, A.J.K. Chua and J.R. Gair, Improving gravitational-wave parameter estimation using Gaussian process regression, Phys. Rev. D 93 (2016) 064001 [arXiv:1509.04066] [INSPIRE].
S. Golchi and R. Lockhart, A Bayesian search for the Higgs particle, arXiv:1501.02226.
M. Frate, K. Cranmer, S. Kalia, A. Vandenberg-Rodes and D. Whiteson, Modeling smooth backgrounds and generic localized signals with Gaussian processes, arXiv:1709.05681 [INSPIRE].
L. Diosan, A. Rogozan and J. Pécuchet, Evolving kernel functions for SVMs by genetic programming, in Sixth International Conference on Machine Learning and Applications (ICMLA 2007), IEEE (2007), p. 19.
W. Bing, Z. Wen-qiong, C. Ling and L. Jia-hong, A GP-based kernel construction and optimization method for RVM, in 2010 the 2nd International Conference on Computer and Automation Engineering (ICCAE), IEEE (2010), p. 419.
D. Duvenaud, J.R. Lloyd, R. Grosse, J.B. Tenenbaum and Z. Ghahramani, Structure discovery in nonparametric regression through compositional kernel search, arXiv:1302.4922.
A.G. Wilson and R. Prescott Adams, Gaussian process kernels for pattern discovery and extrapolation, in Proceedings of the 30th international conference on machine learning, S. Dasgupta and D. McAllester eds., Proc. Machine Learn. Res. 28, PMLR, Atlanta, GA, U.S.A., 17–19 June 2013, p. 1067 [arXiv:1302.4245].
D. Ruppert, M.P. Wand and R.J. Carroll, Semiparametric regression, Cambridge University Press (2003).
ATLAS collaboration, Review of the 13 TeV ATLAS open data release, Tech. Rep. ATL-OREACH-PUB-2020-001, CERN, Geneva, Switzerland (2020).
P.W. Goldberg, K.I. Williams and C.M. Bishop, Regression with input-dependent noise: a Gaussian process treatment, Adv. Neural Inf. Proc. Syst. 10 (1998) 493.
F. Garwood, Fiducial limits for the Poisson distribution, Biometrika 28 (1936) 437.
L. Demortier, Interval estimation, in Data analysis in high energy physics: a practical guide to statistical methods, O. Behnke, K. Kroninger, G. Schott and T. Schorner-Sadenius eds., Wiley-VCH, Berlin, Germany (2013), p. 107.
G. Schwarz, Estimating the dimension of a model, Annals Statist. 6 (1978) 461 [INSPIRE].
H. Akaike, A new look at the statistical model identification, IEEE Trans. Automat. Contr. 19 (1974) 716.
T. Hastie, R. Tibshirani and J. Friedman, The elements of statistical learning: data mining, inference, and prediction, Springer (2001).
T. Hastie and R. Tibshirani, Generalized additive models, Statist. Sci. 1 (1986) 297.
R. Brun and F. Rademakers, ROOT: an object oriented data analysis framework, Nucl. Instrum. Meth. A 389 (1997) 81 [INSPIRE].
F. James and M. Roos, Minuit: a system for function minimization and analysis of the parameter errors and correlations, Comput. Phys. Commun. 10 (1975) 343 [INSPIRE].
M. Hiabu, E. Mammen and J.T. Meyer, Local linear smoothing in additive models as data projection, Ann. Statist. 17 (1989) 453 [arXiv:2201.10930].
ATLAS collaboration, Recommendations for the modeling of smooth backgrounds, Tech. Rep. ATL-PHYS-PUB-2020-028, CERN, Geneva, Switzerland (2020) [INSPIRE].
W.K. Hastings, Monte Carlo sampling methods using Markov chains and their applications, Biometrika 57 (1970) 97 [INSPIRE].
G.J. Feldman and R.D. Cousins, A unified approach to the classical statistical analysis of small signals, Phys. Rev. D 57 (1998) 3873 [physics/9711021] [INSPIRE].
A.L. Read, Presentation of search results: the CLs technique, J. Phys. G 28 (2002) 2693 [INSPIRE].
D. Foreman-Mackey, D.W. Hogg, D. Lang and J. Goodman, emcee: the MCMC hammer, Publ. Astron. Soc. Pac. 125 (2013) 306 [arXiv:1202.3665] [INSPIRE].
D. Foreman-Mackey, corner.py: scatterplot matrices in python, J. Open Source Softw. 1 (2016) 24.
CMS collaboration, Search for pair-produced three-jet resonances in proton-proton collisions at \( \sqrt{s} \) = 13 TeV, Phys. Rev. D 99 (2019) 012010 [arXiv:1810.10092] [INSPIRE].
CMS collaboration, Search for pair-produced resonances each decaying into at least four quarks in proton-proton collisions at \( \sqrt{s} \) = 13 TeV, Phys. Rev. Lett. 121 (2018) 141802 [arXiv:1806.01058] [INSPIRE].
J.H. Collins, K. Howe and B. Nachman, Anomaly detection for resonant new physics with machine learning, Phys. Rev. Lett. 121 (2018) 241803 [arXiv:1805.02664] [INSPIRE].
T. Heimel, G. Kasieczka, T. Plehn and J.M. Thompson, QCD or what?, SciPost Phys. 6 (2019) 030 [arXiv:1808.08979] [INSPIRE].
P. Jawahar et al., Improving variational autoencoders for new physics detection at the LHC with normalizing flows, Front. Big Data 5 (2022) 803685 [arXiv:2110.08508] [INSPIRE].
O. Amram and C.M. Suarez, Tag N’ Train: a technique to train improved classifiers on unlabeled data, JHEP 01 (2021) 153 [arXiv:2002.12376] [INSPIRE].
A. Hallin et al., Classifying anomalies through outer density estimation, Phys. Rev. D 106 (2022) 055006 [arXiv:2109.00546] [INSPIRE].
M. Farina, Y. Nakai and D. Shih, Searching for new physics with deep autoencoders, Phys. Rev. D 101 (2020) 075021 [arXiv:1808.08992] [INSPIRE].
E. Gross and O. Vitells, Trial factors for the look elsewhere effect in high energy physics, Eur. Phys. J. C 70 (2010) 525 [arXiv:1005.1891] [INSPIRE].
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
ArXiv ePrint: 2202.05856
Rights and permissions
Open Access . This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Gandrakota, A., Lath, A., Morozov, A.V. et al. Model selection and signal extraction using Gaussian Process regression. J. High Energ. Phys. 2023, 230 (2023). https://doi.org/10.1007/JHEP02(2023)230
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/JHEP02(2023)230