Search for resonant WZ production in the fully leptonic final state in proton–proton collisions at $\mathbf {\sqrt{s} = 13}$ TeV with the ATLAS detector

Abstract

A search for a WZ resonance, in the fully leptonic final state (electrons or muons), is performed using 139 fb$^{-1}$ of data collected at a centre-of-mass energy of 13 TeV by the ATLAS detector at the Large Hadron Collider. The results are interpreted in terms of a singly charged Higgs boson of the Georgi–Machacek model, produced by WZ fusion, and of a Heavy Vector Triplet, with the resonance produced by WZ fusion or the Drell–Yan process. No significant excess over the Standard Model prediction is observed and limits are set on the production cross-section times branching ratio as a function of the resonance mass for these processes.

1 Introduction

Searches for diboson resonances provide an essential test of theories of electroweak symmetry breaking beyond the Standard Model (BSM): new charged scalar diboson resonances arise in various models with an extended Higgs sector [1,2,3] and vector resonances are predicted in various BSM scenarios [4,5,6,7,8,9]. In this article, a search for a WZ resonance produced via either the Drell–Yan process or vector-boson fusion (VBF) is conducted in the fully leptonic decay channel $\ell \nu \ell \ell$ ($\ell = e ~\textrm{or} ~\mu$) in proton–proton (pp) collisions. The pp collision data, with an integrated luminosity of 139 $\hbox {fb}^{-1}$, were collected by the ATLAS detector [10] at the Large Hadron Collider (LHC) at a centre-of-mass energy of $\sqrt{s} = 13$ TeV.

In the Minimal Supersymmetric Standard Model the tree-level coupling of the charged Higgs boson to WZ is loop-induced [11], and therefore strongly suppressed compared to fermionic couplings. Tree-level coupling to massive vector bosons, is, however, present in extensions of the Standard Model (SM) with higher-isospin scalar fields [12,13,14]. In this article, the Georgi–Machacek (GM) model [15, 16] is used as a benchmark. Because it preserves custodial symmetry at tree level, it is not strongly constrained [17]. The GM model extends the Higgs sector of the Standard Model by including one real and one complex triplet. A parameter, $\sin \theta _H$, representing the mixing of the vacuum expectation values, determines the contribution of the triplets to the masses of the W and Z bosons. The physical scalar states are organized into different custodial multiplets: a fiveplet $(H_5^{++},H_5^+,H_5^0,H_5^-,H_5^{--})$ that is fermiophobic but couples to W and Z bosons, a triplet, and two singlets, one of which is identified as the observed $125~\textrm{GeV}$ Higgs boson with SM properties. Single production of $H^{\pm }_5$ occurs by vector-boson fusion and, in this analysis, the assumption that the triplet states are heavier than the fiveplet scalars implies that it can only decay to $W^\pm Z$. The cross-section is proportional to $\sin ^2 \theta _H$. The singly charged members of this fiveplet are the object of the present search in the VBF channel.

Parameterized Lagrangians [18,19,20] incorporating a Heavy Vector Triplet (HVT) allow the results of searches for vector resonances to be interpreted in a generic way. In this article, a simplified phenomenological Lagrangian [19] is used. The new heavy vector resonance, $W'$, couples to the Higgs field and longitudinally polarized SM gauge bosons by virtue of the equivalence theorem [21], and this coupling is parameterized by the product of two parameters $g_Vc_H$. It couples to the fermions via the combination $(g^2/g_V)c_F$, where g is the SM SU(2) gauge coupling. The parameter $g_V$ represents the typical strength of the vector-boson interaction, while the parameters $c_H$ and $c_F$ are expected to be of the order of unity in most models. The vector-boson scattering process, $pp \rightarrow W' jj \rightarrow WZ jj$, is only sensitive to the gauge boson coupling and, in this case, the benchmark model used to interpret the results assumes no coupling of the heavy vector resonance to fermions.

In nearly all of the parameter space explored in the present analysis and for both benchmark models, the intrinsic width of the resonance is below 4%, which is less than the experimental resolution. Results are provided for the VBF and Drell–Yan production modes separately for the HVT process, neglecting possible signal leakage between them since the VBF contribution is quite small relative to Drell–Yan, always below 1%, and since the VBF benchmark model considered here assumes no coupling to fermions. Representative Feynman diagrams for the production and decay of the heavy resonances searched for in the present analysis are shown in Fig. 1.

Fig. 1

Representative Feynman diagrams for heavy resonance production and decay to WZ bosons a HVT $W'$ production via Drell–Yan, b HVT $W'$ production via vector boson fusion and c GM $H^{\pm }_5$ production via vector boson fusion. The subsequent decays to the $\ell ^{+}\ell ^{-}\ell ^{\pm }\nu$ are also shown

Searches for a $W'$ in an extended gauge model, decaying to WZ in the fully leptonic mode, at $\sqrt{{s}}=8~\textrm{TeV}$ with 20 fb$^{-1}$ of data have been performed by the ATLAS [22] and CMS [23] Collaborations. The present analysis extends searches for resonant WZ production, performed by ATLAS in Run 2 of the LHC using pp collision data at $\sqrt{{s}}=13~\textrm{TeV}$ [24], with 36 $\hbox {fb}^{-1}$ of integrated luminosity.

Exclusion limits from searches of diboson resonances with different final states are summarized in Refs. [25,26,27]. The results from searches for heavy VV and VH vector resonances ($V = W$ or Z) and their combination, based on Run 1 data and on Run 2 data in the fully hadronic (qqqq), semileptonic ($\ell \nu qq, \ell \ell qq, \nu \nu qq$), and fully leptonic ($\ell \ell \ell \ell$, $\ell \nu \ell \ell$, $\ell \ell \nu \nu$) final states are given in Refs. [28,29,30,31,32,33]. The various decay channels generally differ in sensitivity in different mass regions. The fully leptonic channel is found to be more sensitive to resonances with mass below $\sim$ 1 TeV because of the low background, in spite of the low branching ratio. For the VBF process, the present analysis aims to complement previous explorations of the HVT phase space since other channels are mostly insensitive when the coupling of the heavy vectors to fermions is close to zero.

Limits on the GM model have also been set, based on an analysis of opposite-charge WW production by ATLAS [34], using data at $\sqrt{s}=13~\textrm{TeV}$. Searches [35, 36] by the CMS Collaboration for a singly-charged and a doubly-charged Higgs boson, produced via VBF and decaying respectively into WZ and WW in the fully leptonic mode, using an integrated luminosity of 137 fb$^{-1}$, have yielded limits on the coupling parameter of the GM model, assuming degenerate masses of $H_5^{\pm }$ and $H_5^{\pm \pm }$. Upper bounds at 95% confidence level (CL) on $\sin \theta _H$ vary between $\sim 0.2$ and 0.55 in the mass range 200–2000 GeV. In the present analysis, in addition to the larger data set, several improvements relative to the previously published analysis [24] have been implemented, most notably the implementation of multivariate techniques for the VBF signal selection.

2 The ATLAS detector

The ATLAS detector [10] has a cylindrical geometry with a nearly $4\pi$ coverage in solid angle.¹ The inner detector (ID), consisting of silicon pixel, silicon microstrip and transition radiation detectors, is surrounded by a thin superconducting solenoid providing a 2 T axial magnetic field. It allows precise reconstruction of tracks from charged particles and measurement of their momenta up to a pseudorapidity of $|\eta | = 2.5$. High-granularity lead/liquid-argon (LAr) sampling electromagnetic and steel/scintillator-tile hadron calorimeters, at larger radius, provide energy measurements in the central pseudorapidity range $|\eta | < 1.7$. In the endcap and forward regions, LAr calorimeters for both the electromagnetic and hadronic energy measurements extend the region of angular acceptance up to $|\eta | = 4.9$. Outside the calorimeters, the muon spectrometer incorporates multiple layers of trigger and tracking chambers in a magnetic field produced by a system of superconducting toroid magnets, enabling an independent precise measurement of muon track momenta for $|\eta |<2.7$. The ATLAS trigger system consists of a hardware-based level-1 trigger followed by a software-based high-level trigger [37]. An extensive software suite [38] is used in the reconstruction and analysis of real and simulated data, in detector operations, and in the trigger and data acquisition systems of the experiment.

3 Data and Monte Carlo samples

The data used were collected from 2015 to 2018 with the ATLAS detector from pp collisions at a centre-of-mass energy of $13~\textrm{TeV}$ at the LHC, and initially selected by requiring that a set of quality criteria for detector and data conditions be satisfied [39].

Events were required to pass combinations of single-electron or single-muon triggers [40, 41]. The transverse momentum (${{p}_{\textrm{T}}}$) thresholds of the leptons in 2015 were 24 GeV for electrons and 20 GeV for muons, with both satisfying a loose isolation requirement based only on ID track information. Due to the higher instantaneous luminosity in 2016–2018 the trigger threshold was increased to 26 GeV for both the electrons and muons, and tighter isolation requirements were applied. An additional electron (muon) trigger with a ${{p}_{\textrm{T}}}$  threshold of 60 (50) GeV and no isolation requirement, and a single-electron trigger requiring ${{p}_{\textrm{T}}}$  > 120 GeV with less restrictive electron identification criteria, were used to increase the selection efficiency, which reached almost 100%, relative to the offline selections [40, 41]. With these conditions, the integrated luminosity used in this analysis is 139 $\hbox {fb}^{-1}$.

Simulated signal events and background processes with prompt leptons were used to model the benchmark physics processes and optimize the selection cuts. They were produced by Monte Carlo (MC) generators with the detector response modelled by the Geant toolkit [42, 43] integrated into the ATLAS simulation infrastructure. For some samples, the calorimeter response is obtained from a fast parameterized detector simulation [44], instead of full simulation by Geant. The effect of multiple interactions in the same and neighbouring bunch crossings (pile-up) was modelled by overlaying the simulated hard-scattering event with inelastic pp events generated with Pythia 8.186 [45] using the NNPDF2.3LO set of parton distribution functions (PDF) [46] and the A3 set of tuned parameters (tune) [47]. The distribution of the number of pile-up events reproduces the bunch structure and the average number of interactions per bunch crossing in the various run periods. A pileup weight is defined, which is applied to Monte Carlo to correct for the difference between the distribution of average number of interactions used to produce the sample and that measured for the recorded data. For all samples, except those generated with Sherpa [48], the EVTGEN 1.2.0 program [49] was used to simulate the properties of the b- and c-hadron decays.

The GM VBF benchmark signal samples, $pp \rightarrow H_5^\pm jj \rightarrow W^\pm Z jj \rightarrow \ell ^\pm \nu \ell ^+\ell ^- jj$, vetoing W or Z bosons in the s-channel, were produced with MadGraph 2.7.2 [50] at next-to-leading order (NLO) in QCD [1, 51]; the generator is referred to as MADGRAPH   hereafter. The signal simulation is produced for the mass range 200 GeV to 1 TeV in the $H_5$-plane defined in Refs. [1, 52], using the tool GMCALC [53]. The parameter $\sin \theta _H$ was set to 0.5 for masses up to 800 GeV and 0.25 for higher masses to be compatible with present constraints [52]. The matrix element calculation employed the NNPDF3.0NLO [54] set of PDFs. Events were interfaced to Pythia 8.186 for the modelling of the parton shower, hadronization, and underlying event, using the A14 tuning parameters [55] and with the dipole recoil shower scheme to prevent the generation of excess central jet radiation [56]. For these samples, a minimum ${{p}_{\textrm{T}}}$  of $15~\textrm{GeV}$ ($10~\textrm{GeV}$) for the jets (leptons) was required during event generation. The signal samples were produced in $25~\textrm{GeV}$ mass steps up to $600~\textrm{GeV}$ and $100~\textrm{GeV}$ mass steps up to $1~\textrm{TeV}$.

Two benchmark models of HVT production via the Drell–Yan process, $q q'\rightarrow W' \rightarrow WZ \rightarrow \ell \nu \ell \ell$, are used to interpret the results. Model A is typical of weakly coupled vector resonances arising from an extension of the SM gauge group [57] with an additional SU(2) symmetry, and the branching ratios to fermions and gauge bosons are comparable. Model B is representative of an HVT produced in a strongly-coupled scenario, as in a Composite Higgs model [58] with suppressed fermionic couplings. The parameter $g_V$ was set to 1 for Model A and to 3 for Model B. For both models, $c_F$ is set to 1 and is assumed to be the same for all types of fermions. The simulated signal samples for Model A were generated at leading order (LO) in QCD with MadGraph 2.6.5 using the model file provided by the authors of Ref. [19]. The parton-level simulated events were hadronized with Pythia 8.186, using the NNPDF23_lo_as_0130_qed PDF set and A14 tune. The signal samples were produced for vector resonances with masses ranging from 250 to $5~\textrm{TeV}$, in steps of $25~\textrm{GeV}$ below $600~\textrm{GeV}$, $100~\textrm{GeV}$ between $600~\textrm{GeV}$ and $2~\textrm{TeV}$, $200~\textrm{GeV}$ from $2~\textrm{TeV}$ to $3~\textrm{TeV}$ and $500~\textrm{GeV}$ above. For interpretation in terms of Model B, the Model A simulation is used and the cross-sections were simply scaled. This is justified since the intrinsic resonance width remains well below the experimental resolution and the angular distributions are the same for both models.

For the VBF production mode of heavy vector resonances, which is expected to have a low cross-section, the benchmark model used is also based on the HVT parameterization. The coupling parameters $g_V$ and $c_H$ are set to 1 and all other couplings of the heavy triplet, including $c_F$, are set to 0 in order to maximize the VBF contribution. The simulated signal samples were generated at LO in QCD with MadGraph 2.6.5 using the model file provided by the authors of Ref. [19]. The parton-level simulated events were hadronized with Pythia 8.186, using the NNPDF23_lo_as_0130_qed PDF set and A14 tune. A dijet invariant mass of at least $150~\textrm{GeV}$ is required in this case at event generation. The simulation samples were generated for masses ranging from $300~\textrm{GeV}$ to $2~\textrm{TeV}$, in steps of $25~\textrm{GeV}$ ($100~\textrm{GeV}$) up to (beyond) $600~\textrm{GeV}$.

The background sources include processes with two or more electroweak gauge bosons, namely VV and VVV ($V=Z$, W) as well as processes with top quarks, such as $t\bar{t}$, $t\bar{t}V$, and single top-quark, and processes with gauge bosons produced in association with jets or photons.

The dominant background for this search is the SM QCD mediated WZ process, referred to as WZ-QCD here. It includes processes up to order four in the electroweak coupling constant, $\alpha _{EW}$, and is modelled using Sherpa 2.2.2 [48]. The WZ sample includes up to one jet calculated at NLO in QCD, while second and third jets were calculated at LO in QCD and merged with the parton shower. In order to estimate an uncertainty due to generator and parton shower modelling, an alternative NLO WZ-QCD sample was produced using MadGraph 2.6.5 with FxFx merging [59] of up to two extra jets, using the PDF set NNPDF30_nlo_as_118. The hadronization was performed with Pythia 8.186 with the A14 tune. A sample of the purely electroweak process $WZjj \rightarrow \ell \nu ~\ell \ell ~jj$, including processes of order six in $\alpha _{EW}$ ( WZ-EWK), was generated separately with MadGraph 2.7.3 together with Pythia 8.244 using the A14 tune [55] and the NNPDF3.0NLO [54] PDF set. To estimate an uncertainty due to the parton shower modelling, a sample using the same MadGraph 2.7.3 matrix element but Herwig 7.2.1 for the parton shower was produced. According to the SM, a small amount of interference occurs between electroweak and QCD WZ production. This was modelled with MadGraph 2.7.3 + Pythia 8.244 using the A14 tune [55] and the NNPDF3.0NLO [54] PDF set and combined with the simulated WZ-EWK sample.

Samples of $q\bar{q}\rightarrow ZZ \rightarrow 4\ell$, $q\bar{q}\rightarrow ZZ \rightarrow \ell \ell ~\nu \nu$ and triboson events were generated with Sherpa 2.2.2 [60] using matrix elements at NLO accuracy in QCD for up to one additional parton and at LO accuracy for up to three additional parton emissions. The simulation included off-shell effects and Higgs boson contributions. The purely electroweak process $q\bar{q}\rightarrow ZZjj \rightarrow 4\ell jj$ and the $gg\rightarrow ZZ$ process were also generated with Sherpa 2.2.2 [60]. The LO-accurate matrix elements were matched to a parton shower based on Catani–Seymour dipole factorization [61, 62] using the MEPS@LO prescription [63,64,65,66]. Samples were generated using the NNPDF3.0NNLO PDF set [54], and SHERPA parton-shower parameter values.

The $t\bar{t}V$ processes were modelled using the MadGraph 2.3.3 [50] generator at NLO in QCD with the NNPDF3.0NLO [54] PDF set. The events were interfaced to PYTHIA 8.210 [67] using the A14 tune and the NNPDF2.3 LO [54] PDF set.

Finally, samples of SM backgrounds with at least one misidentified or non-prompt lepton, including $Z\gamma$, $W\gamma$, Drell–Yan $Z\rightarrow \ell \ell ,~W\rightarrow \ell \nu$ as well as top-quark pairs and single top-quark have been generated to assist in estimating the fake/non-prompt lepton background. Events with $V\gamma$ in the final state were simulated with the SHERPA [2.2.4] [60] generator. Matrix elements at LO accuracy in QCD for up to three additional parton emissions were matched and merged with the SHERPA parton shower [61,62,63,64,65,66]. The samples were generated using the NNPDF3.0NNLO PDF set [54], along with the dedicated set of tuned parton-shower parameters developed by the SHERPA authors. Drell–Yan $Z\rightarrow \ell \ell$ and $W\rightarrow \ell \nu$ were produced with Powheg box v1 generator [68,69,70,71] at NLO accuracy for the hard-scattering processes of W and Z boson production and decay in the electron, muon, and $\tau$-lepton channels. The events were interfaced to PYTHIA 8.186 [45] for the modelling of the parton shower, hadronization, and underlying event, with parameters set according to the AZNLOtune [72]. The CT10NLO PDF set [73] was used for the hard-scattering processes, whereas the CTEQ 6L1 PDF set [74] was used for the parton shower. The effect of QED final-state radiation was simulated with (PHOTOS)++ 3.52 [75, 76]. For top-quark pairs and single top-quark productions the Powheg box v2 [68,69,70, 77] generator was used at NLO with the NNPDF3.0NLO [54] PDF set. The events were interfaced with PYTHIA 8.230 [67] using the A14 tune [55] and the NNPDF2.3LO PDF set.

SM backgrounds with Higgs bosons ($H, t\bar{t}H, VH$) contribute less than 0.1% of the total background because of the low cross-section and the requirement of a well-reconstructed leptonically decaying Z boson. These backgrounds were neglected.

4 Object reconstruction and identification

Electron candidates are reconstructed from energy deposits in the electromagnetic calorimeter which are matched to a well-reconstructed ID track [78]. Only electrons with transverse energy $E_{\textrm{T}}>$7 GeV and within the pseudorapidity range of $|\eta |<2.47$, excluding the barrel–endcap transition region $1.37<|\eta |< 1.52$ are considered. Muons are identified either by matching muon spectrometer tracks with tracks in the ID or by using the calorimeter-based muon identification [79, 80]. They are required to have transverse momentum ${{p}_{\textrm{T}}}>5$ GeV  (${{p}_{\textrm{T}}}>15$ GeV if calorimeter tagged) and pseudorapidity $|\eta |<2.7$.

Identification and isolation criteria, either ‘loose’, ‘medium’ or ‘tight’ as described in Refs. [78, 79], are applied to electron and muon candidates. Identification criteria are based on shower shapes and track parameters for electrons, and on track parameters for muons. The isolation criteria use information about ID tracks and calorimeter energy deposits in a fixed cone of size $\Delta R =~0.2$ around each lepton. Four lepton categories are designed using the identification and isolation criteria: Baseline electrons and muons are required to satisfy ‘loose’ identification and isolation criteria (for muons with ${{p}_{\textrm{T}}}>300$ GeV  the dedicated ‘High ${{p}_{\textrm{T}}}$ identification’ is required [80]). The Loose, Tight Z and Tight W leptons are defined as subsets of the Baseline lepton selection with ${{p}_{\textrm{T}}}>25$ GeV. For the Tight Z leptons the ‘medium’ identification and ‘tight’ isolation criteria are applied, while for Tight W leptons the ‘tight’ identification and ‘tight’ isolation criteria are applied. The tighter identification and isolation requirement applied to the lepton of the W candidate is motivated by lower background rates for leptons from the Z candidate, which are well constrained by the requirement on their invariant mass.

Electron and muon candidates are required to originate from the primary vertex. The primary vertex is defined, using tracks with ${{p}_{\textrm{T}}}>$ 500 MeV, as the vertex candidate with the highest $\sum {{p}_{\textrm{T}}}^2$ of its associated tracks. The transverse impact parameter of the track ($d_{0}$) is calculated relative to the beam line. For all four lepton categories, the longitudinal impact parameter, $z_0$ (the difference between the value of z at the point of the track where $d_0$ is defined and the longitudinal position of the primary vertex), is required to satisfy $|z_0 \cdot \textrm{sin}\theta |<0.5$ mm, where $\theta$ is the polar angle of the track momentum at the reference point. Furthermore, for the Loose, Tight Z and Tight W leptons the significance of the transverse impact parameter of the track, $|d_{0}/\sigma _{d_0}|$, where $\sigma _{d_0}$ stands for the resolution of $d_{0}$, must be smaller than 3.0 for muons and less than 5.0 for electrons.

Jets are based on particle-flow objects built from noise-suppressed positive-energy topological clusters of cells in the calorimeter and reconstructed tracks [81]. The anti-$k_t$ algorithm [82, 83] with a radius parameter of $R=~0.4$ is used. For jets, the main backgrounds are either beam-induced, due to proton collisions upstream of the interaction point, from cosmic-ray showers or highly coherent calorimeter noise. These jets are considered ‘unclean’ and nearly all are rejected by applying a set of quality criteria. Furthermore, to mitigate contamination from the pile-up, a jet vertex tagger [84, 85], using information about tracks associated with the primary vertex and pile-upvertices, is applied to jets with ${{p}_{\textrm{T}}}< 60~\textrm{GeV}$ and $|\eta |<2.4$. In the forward region, pile-upjet tagging that exploits jet shapes and topological jet correlations in pile-upinteractions is applied to jets with ${{p}_{\textrm{T}}}< 120~\textrm{GeV}$ and $2.5<|\eta |<4.5$ [85].

The flavour of jets is determined using a deep-learning neural network, DL1r [86, 87]. The DL1r b-tagging is based on distinctive features of b-hadron decays in terms of the impact parameters of the tracks and the displaced vertices reconstructed in the inner detector. The b-tagging algorithm has an efficiency of 85% in simulated $t\bar{t}$ events, a light-flavour jet misidentification probability of 3% and a c-jet misidentification probability of about 33%.

Two levels of jet selections are used: the Baseline jets have ${{p}_{\textrm{T}}}> 30$ GeV and $| \eta |<4.5$, while for VBF jets, which are a subset of Baseline jets, the pile-upremoval using the jet vertex tagger and a b-tagging veto are applied, as described above, since they are mostly forward jets and not initiated by b-quarks.

To avoid cases where the detector response to a single physical object is reconstructed as two different final-state objects, an overlap-removal procedure is applied to the Baseline selected leptons and jets. If two electrons share the same track then the lower-${{p}_{\textrm{T}}}$ electron is discarded. Electrons that share the same track as a selected muon with a muon spectrometer track are also discarded; but in the case of a calorimeter-tagged muon, it is the muon which is rejected. A jet is removed if its separation from an electron satisfies $\Delta R<$ 0.2; the electron is removed if the separation satisfies 0.2 $<\Delta R<$ 0.4. For nearly collinear muons and jets, the jet is removed if it is separated from the muon by $\Delta R<$ 0.2 and if it has less than three tracks, or if the energy and momentum differences between the muon and the jet are small; otherwise the muon is removed if the separation satisfies $\Delta R<$ 0.4.

The missing transverse momentum, $E_{\textrm{T}}^{\textrm{miss}}$, in an event is calculated as the magnitude of the negative vectorial sum of the transverse momenta of all Baseline selected and calibrated physics objects that can be matched to the primary vertex. A component called the “soft term” is calculated from the residual tracks that originate from the primary vertex but are not associated with any other object and is added to the $E_{\textrm{T}}^{\textrm{miss}}$ calculation [88].

5 Event selection

In this search all final states with three charged leptons (e or $\mu$) and missing transverse momentum from WZ leptonic decays are considered. The search begins with a WZ baseline selection, and two selections are defined in order to build signal regions (SRs) targeting the Drell–Yan and VBF productions modes. A cut-based selection is used to build the Drell–Yan signal region, while for the VBF selection, an artificial neural network (ANN) was trained. An alternative, cut-based selection for the VBF is also presented in the Appendix. The invariant mass of the WZ candidates, m(WZ), built with the leptons and $E_{\textrm{T}}^{\textrm{miss}}$  is used as the discriminating variable. A summary of all the selections used to define the analysis signal regions (SRs) and control regions (CRs) can be found in Table 1.

Table 1 Summary of the event selections for signal and control regions. Definitions of some variables used in this table can be found in Sects. 5.2 and  5.3

5.1 Baseline selection of WZ events

The baseline selection is a set of event criteria applied to data and all simulated samples before defining more specific analysis regions. First, there is a requirement of good quality for the recorded events, based on the working conditions of all subdetectors. Events are vetoed if they have one or more unclean jets. All events are required to contain a primary vertex with at least two associated tracks.

Events are required to contain exactly three leptons meeting the Loose selection criteria defined in Sect. 4. In order to reduce the ZZ background, events with four or more leptons meeting the Baseline criteria are vetoed. To ensure that the trigger efficiency is well determined, at least one of the three candidate leptons must be trigger-matched and is required to have ${{p}_{\textrm{T}}}$>27 GeV. A Z candidate must be present. It is defined by two leptons of the same flavour and opposite charge with an invariant mass that is consistent with the Z boson pole mass ($m_Z$): $|m_{\ell \ell } - m_Z| < 20$  GeV. If there is more than one pair of leptons that can form a Z candidate, the one with invariant mass closest to the Z boson pole mass is chosen. The third lepton is then taken as the W boson lepton candidate. The leptons assigned to the W and Z candidates are then required to satisfy the Tight W or Tight Z selection criteria defined in Sect. 4. Finally, the missing transverse momentum in the event is required to be greater than 25 GeV.

To reconstruct the four-vector of the W boson, the $E_{\textrm{T}}^{\textrm{miss}}$of the event is assumed to be due to the neutrino. The longitudinal component $p_z(\nu )$ of the neutrino momentum is estimated by constraining the invariant mass of the $\ell \nu$ system to be the pole mass of the W boson, where the charged lepton is the one assigned to the W candidate. A quadratic equation leads to two solutions. If they are real, the one with the smaller magnitude of $|p_z(\nu )|$ is chosen, otherwise, the real part is chosen. The choice of the solution was optimized using truth information. The invariant mass of the WZ system is then calculated.

5.2 Drell–Yan process selection

For a heavy resonance produced essentially at rest in the s-channel, it is expected that the selected W and Z bosons have transverse momenta close to 50% of the resonance mass. A boson ${{p}_{\textrm{T}}}$ to resonance mass ratio variable is therefore defined as the ratio ${{p}_{\textrm{T}}}(V)/m(WZ)$ of the boson transverse momentum to the WZ invariant mass. To reduce the contribution from non-resonant WZ production, events passing the WZ preselection are required to have a boson ${{p}_{\textrm{T}}}$ to resonance mass ratio greater than 0.35 for both bosons. The combined detector acceptance and signal selection efficiency ($A\times \epsilon$) of the Drell–Yan HVT $W'$ selection, relative to the generated signal events, is shown in Fig. 2. There, decays of W and Z bosons into all flavours of leptons are included at event generation. The $A\times \epsilon$ values decrease for resonance masses above approximately 2 TeV due to the collinearity of electrons from highly boosted $Z\xrightarrow []{}ee$ decays, for which the lepton isolation is less efficient.

Fig. 2

The acceptance (A) times efficiency ($\epsilon$) of the HVT $W'$ in the Drell–Yan signal region for different mass points and for the individual channels $\mu \nu \mu \mu$, $e\nu ee$, $\mu \nu ee$, $e\nu \mu \mu$, and the sum of all channels. The uncertainty includes both the statistical and experimental systematic components

5.3 Vector boson fusion process selection

The VBF process ($pp \rightarrow W' j j \rightarrow WZ j j$) is characterized by the presence of two jets with a large rapidity gap resulting from quarks from which a vector boson has been radiated. To select the signal events, an artificial neural network with a binary classification task is used: events are categorized as belonging either to a VBF process or to the background. The ANN training is implemented using the Keras package [89] running on top of the TensorFlow package [90]. An ANN training region is defined by requiring events to have at least a pair of jets satisfying the VBF jets selection, and from those, the pair with the highest-${{p}_{\textrm{T}}}$ is required to have an invariant $m_{\textit{jj}}>$100 GeV. The ANN is trained in this region with simulated $H^{\pm }_5$ events as signal, against the SM WZ-EWK and WZ-QCD events as background. The $H^{\pm }_5$ simulation is used for the training because the kinematic variables show very similar distributions for the GM and HVT benchmark signals and the training yielded similar results.

In order to minimize the statistical uncertainty, a 4-fold cross-validation technique was applied. A rectified linear unit, or a ReLU, was used as an activation function at each node [91]. The space of hyperparameters was scanned and a final set was chosen to ensure optimal performance of the network. The training was performed with 100 epochs, Nesterov’s momentum of 0.7 [92], and two hidden layers of 45 neurons each. To avoid overfitting, a regularization technique was employed. For each input sample, a hidden layer was randomly removed with a probability parameter (dropout) of 0.2, allowing for a noisy training process.

The distributions of the loss function and of accuracy vs epochs were monitored for the training and validations sets and no sign of overtraining was observed.

Table 2 Variables used for ANN training

The input variables² used for the ANN optimization are listed in Table 2. These were chosen on the basis of their impact in the training and such that highly correlated variables are not used simultaneously. The loss in expected significance when adding or exchanging some of the variables was evaluated for each set of variables until the optimal set was found.

Fig. 3

Distribution of the ANN score in the WZ-QCD VBF control region and the VBF signal region. The background predictions are obtained from a background-only simultaneous fit to the VBF signal region, the WZ-QCD VBF and ZZ VBF control regions as described in Sect. 8. The uncertainty in the total background prediction, shown as a hashed area, combines statistical and systematic contributions. The distributions for the HVT VBF model $W'$ and GM $H_{5}^{\pm }$ simulations are shown normalized to the data integral. The vertical dotted line shows the threshold value for the ANN output score used to define the VBF signal region

All the mass samples of simulated $H^{\pm }_5$ GM events are used simultaneously to define the “signal” for the training. After training, the threshold for the ANN output score is chosen in such a way that it maximizes the significance for the lowest mass point (200 GeV). The advantage of this approach is that it greatly reduces the training effort and a single signal region can be used. It was verified that the alternative of using one ANN training per mass point does not significantly improve the performance. The training is then applied to both GM and HVT Model samples. A minimum value of 0.82 on the ANN output maximizes the significance and is chosen to define the signal region. After all selection cuts are applied the VBF signal region effectively starts at $m_{\textit{jj}}>500~\textrm{GeV}$. This signal region was blinded until the background and its uncertainties in the control regions had been evaluated (Sect. 6).

The distribution shapes and correlations of all input variables to the ANN were found to be well modelled by MC simulation in the WZ-QCD control region (see Sect. 6 for definition). This is exemplified by the good description of the ANN output score distribution of data in the WZ-QCD control region and VBF signal region shown in Fig. 3.

The acceptance times efficiency ${\mathcal {A}}\times \epsilon$ of the ANN-based VBF selection as a function of the mass of the VBF $H^{\pm }_{5}$ and of the HVT $W'$ boson, relative to the generated signal events, is shown in Fig. 4. There, decays of W and Z bosons into all flavours of charged leptons are included at event generation. For the $H^{\pm }_{5}$ and the HVT bosons the ${\mathcal {A}}\times \epsilon$ value is in the range 2–12% and 2–5% respectively for resonance masses ranging between 200 and 1000 GeV, the difference being due, with approximately equal importance, to the generator-level selection and the different angular distributions of the final products.

Fig. 4

The acceptance (A) times efficiency ($\epsilon$) of VBF $H^{\pm }_{5}$ and HVT $W'$ selection after the ANN-based VBF selection at different mass points for the individual channels $\mu \nu \mu \mu$, $e\nu ee$, $\mu \nu ee$, $e\nu \mu \mu$, and the sum of all channels. The uncertainty includes both the statistical and experimental systematic components

6 Background estimation

The backgrounds are classified into two groups: the irreducible backgrounds where all reconstructed lepton candidates are prompt (arise from the primary process) and the reducible backgrounds where at least one of the lepton candidates is not prompt. Non-prompt leptons are also referred to as “fake/non-prompt” leptons.

The contributions from the irreducible backgrounds WZ-QCD, WZ-EWK, ZZ, VVV and $t\bar{t}V$ are estimated using MC simulation. The normalizations of WZ-QCD and ZZ are constrained by data in a simultaneous fit using the signal region and dedicated control regions. Each signal region has two associated CRs designed to match that particular SR’s event topology and jet multiplicity, as summarized in Table 1.

The dominant source of irreducible background is QCD-mediated production of WZ dibosons. Two CRs are created to constrain it. One is referred to as the WZ-QCD Drell–Yan CR and is dedicated to the Drell–Yan analysis. It is defined by selecting the subsample of WZ events that fulfill all the Drell–Yan event selection except the boson ${{p}_{\textrm{T}}}$ to resonance mass ratio. The lower bound on the boson ${{p}_{\textrm{T}}}$ to resonance mass ratio is set to 0.1 to bring this CR close to the signal region. The second CR, referred to as the WZ-QCD VBF CR, is dedicated to the VBF analyses. It is defined by selecting from the WZjj subsample the events that fail the ANN output score requirement, and have $m_{\textit{jj}}>500$ GeV. The high $m_{\textit{jj}}$ requirement is applied in order to match the signal region’s event topology. In both CRs the WZ-QCD contribution is around 80%.

To extract the ZZ background normalization, two ZZ-enriched control regions are defined after applying the WZ event preselection described in Sect. 5.1. The presence of at least a fourth lepton candidate satisfying the Baseline identification criteria is required and no requirement on the missing transverse momentum is applied. This region is used for the Drell–Yan process analysis and is referred to as the ZZ Drell–Yan CR. For the VBF selection, the events in the ZZ VBF CR must have, in addition, at least two VBF tagged-jets.

The QCD-mediated production of ZZ events represents 91% (80%) of the ZZ Drell–Yan CR (ZZ VBF CR), while the contribution of ZZ events from electroweak-mediated production ZZ is small. In the following the sum of the two components is referred as the ZZ background.

To validate the modelling of the $t\bar{t}V$ background, a dedicated validation region is built by requiring the WZjj events to have at least one b-tagged jet. Since no significant data mis-modelling was observed, the m(WZ) shape and normalization of this background are taken from simulation.

The reducible backgrounds originate from Drell–Yan $Z\rightarrow \ell \ell ,~W\rightarrow \ell \nu$, $Z\gamma$, $t\bar{t}$, Wt and WW processes where jets or photons were misidentified as leptons. For both analysis regions the normalizations of the reducible backgrounds are estimated using a data-driven method. The method is based on a global matrix which exploits differences between the characteristics of real and fake/non-prompt leptons on a statistical basis. Details of the method can be found in Ref. [93]. The shape in the Drell–Yan analysis is obtained from the data-driven method. In the VBF analysis, due to the fewer data events, the shapes are taken from simulation.

7 Systematic uncertainties

Systematic uncertainties from the theoretical modelling and the object and event reconstruction have an impact in the signal and control regions used. The search sensitivity is then affected by their effects on background estimates, signal acceptance, and the shape of the distributions of the invariant mass discriminant. Depending on the nature of the uncertainty these can be classified into two groups: (a) theoretical uncertainties associated with the MC modelling of both the background and signal processes and (b) experimental uncertainties related to the detector and reconstruction performance. The uncertainties and the methods used to evaluate them are discussed below. Unless explicitly stated, the uncertainties quoted are the uncertainties in the quantities themselves, not their impact on the search sensitivity.

7.1 Theoretical uncertainties

Systematic uncertainties in the theoretical modelling by the event generators used to evaluate the WZ-$\hbox {QCD}$, WZ-$\hbox {EWK}$ and ZZ templates are considered. For the WZ-$\hbox {QCD}$ and ZZ backgrounds that have data-driven normalization only the shape variations of the reconstructed m(WZ) distribution are considered. Uncertainties due to higher-order QCD corrections are evaluated by varying the renormalization and factorization scales independently by factors of two and one-half. For the WZ-$\hbox {QCD}$ background only a small shape effect is observed and used in the fit. For the WZ-$\hbox {EWK}$ background the uncertainties in the m(WZ) shape grow with the mass from 8 to 15%. The uncertainties due to the PDF and the $\alpha _s$ value used in the PDF determination are evaluated using the PDF4LHC prescription [94]. For the WZ-$\hbox {QCD}$ background they are estimated to have a small shape component but are nevertheless included in the fit. For the WZ-$\hbox {EWK}$ they are added in quadrature, and the total uncertainty stays between 5 to 6% in all mass bins for both the Drell–Yan and VBF selections. A modelling uncertainty in the WZ-QCD background template including effects of the parton shower model, is estimated by comparing predictions of the m(WZ) distribution from the Sherpa and MadGraph MC generators. The difference between the two predicted m(WZ) distribution shapes is used as an uncertainty band centred around the nominal Sherpa prediction. A parton shower modelling uncertainty in the WZ-EWK background template is estimated using two MC samples with different parton shower models, Pythia and Herwig. This modelling uncertainty has no effect on the m(WZ) distribution’s normalization at low mass, but grows to 5% at high mass.

For the ZZ background the shape uncertainties originating from the renormalization and factorization scales, as well as from the PDF and the chosen value of $\alpha _s$ are evaluated in a similar way.

The theoretical uncertainties described above are evaluated in all the analysis signal and control regions and treated as uncorrelated across those regions in the statistical analysis.

An uncertainty of 20% is assigned to the $t\bar{t}V$ and VVV cross-sections [95,96,97]. It consists of contributions from PDF uncertainties and QCD scale uncertainties.

Uncertainties in the signal acceptances due to PDF and scale choices are also evaluated. These uncertainties are calculated following the procedure described above, for several resonance mass points, and for each model, production process and decay. The theoretical uncertainties of the HVT signals are evaluated to be less than 20% for all production modes and they are 30% for the GM model.

7.2 Experimental uncertainties

Experimental uncertainties arise from the determination of the luminosity, the lepton trigger efficiency, the reconstruction and identification efficiencies of leptons and jets, and the missing transverse momentum.

The uncertainty in the integrated luminosity is 1.7%. It is derived following a methodology similar to that detailed in Ref. [98], using the LUCID-2 detector for the baseline luminosity measurements [99], and calibrating the luminosity scale using x-y beam-separation scans. A variation in the pile-upreweighting of MC events is included to cover the uncertainty in the ratio of the predicted and measured inelastic cross-sections [100].

Systematic uncertainties affecting the reconstruction and energy calibration of jets are propagated through the analysis. They are the dominant experimental uncertainties in the VBF selection. Those due to the jet energy scale and resolution are obtained from simulations and in situ techniques [101]. The uncertainties in the b-tagging efficiency and the mis-tag rate are also taken into account. The effect of jet uncertainties on the expected number of events ranges up to 15% in the VBF selection.

Uncertainties in the efficiencies of the lepton triggers are found to be negligible. The uncertainties due to the electron and muon reconstruction, identification and isolation requirements are estimated using tag-and-probe methods in $Z\rightarrow \ell \ell$ events in data and simulation [78, 79]. Uncertainties in the lepton energy scale and resolution are also assessed. The impact of lepton uncertainties on the expected number of events is typically below 1%.

The uncertainty in the measurement of missing transverse momentum is estimated by propagating the uncertainties in the transverse momenta of preselected leptons and jets, as well as those in the soft term [88].

An uncertainty in the prediction of the fake/non-prompt background is also taken into account because it affects the shape and normalization of the background distributions. The total uncertainty is about 60% (more than 100%) for the Drell–Yan (VBF) selections. It is larger for the VBF selection because of the higher statistical uncertainty.

8 Results

8.1 Statistical analysis strategy

The WZ invariant mass distribution is used as the discriminating variable. The bin widths were chosen to be comparable to the expected resolution for the resonance model under investigation, and at the same time to optimize the sensitivity of the search while reducing the impact of statistical fluctuations.

A profile-likelihood-ratio test statistic [102] is used to test the compatibility of the background-only hypothesis with the data and to test the signal-plus-background hypothesis. The binned likelihood function is constructed by considering, in each bin, the contributions of the backgrounds and of a hypothetical signal of given strength relative to a benchmark model’s production cross-section. In the absence of an observed signal, exclusion limits on the presence of a signal are then derived using the CLs method [103]. All lepton flavours and data taking periods are combined together for the profile-likelihood-ratio test since there was no significant gain in splitting the samples before the statistical analysis.

Simultaneous maximum-likelihood fits to the observed binned distributions of m(WZ) in the signal regions and their dedicated WZ and ZZ control regions are performed. Separate fits are performed for the Drell–Yan and VBF selections. The normalizations of the WZ-QCD and ZZ contributions are freely floating parameters in these fits and are constrained by the data in both the SRs and dedicated CRs. The ratio of the fitted contributions in the CR and SR is allowed to vary within the theoretical uncertainties. The normalizations and shapes of all other backgrounds are allowed to vary within their uncertainties.

Systematic uncertainties, described in Sect. 7, and their correlations are incorporated in the likelihood as nuisance parameters with Gaussian constraints. Most of the systematic uncertainties are taken to be correlated between the SR and CRs and fit simultaneously in these regions, with the theoretical uncertainties of the ZZ, WZ-QCD and WZ-EWK backgrounds being the only exceptions.

Two fit configurations are used, referred to as the Drell–Yan and VBF configurations. The Drell–Yan fits include the Drell–Yan SR, WZ-QCD CR and ZZ-CR. In the VBF configuration, fits include the VBF-SR, WZjj-QCD CR and ZZjj-CR. Separate fits are performed for the different models tested and for different resonance mass hypotheses. The Drell–Yan configuration is used to search for a $W'$ boson predicted by the HVT benchmark. Two VBF fits are performed using the VBF configuration: one for the search for a VBF-produced $W'$ predicted by the HVT model, and the other for the search for a charged Higgs boson, $H_5^{\pm }$, as predicted by the GM model.

8.2 Data and background comparisons

To test the compatibility of the data and the background expectations, the data are first fit to the background-only hypothesis, separately in the Drell–Yan and VBF configurations.

The post-fit background yields are summarized in Table 3 for the Drell–Yan and VBF signal regions. In both cases the fit is able to adjust the SM ZZ and WZ-QCD background normalizations using the data in signal and control regions. In the Drell–Yan fit, the ZZ background normalization is increased by around 10% while the WZ-QCD background is decreased by 10% relative to the pre-fit predictions. Some mild pulls in the modelling uncertainties by less than one standard deviation from their pre-fit values are visible in the Drell–Yan fit. In the VBF fits, the normalization of the ZZ background is consistent with the pre-fit value while the WZ-QCD background is reduced by around 30%. Apart from the mild pulls in the Drell–Yan signal region, none of the other nuisance parameters are significantly pulled or constrained relative to their pre-fit values in any of the background-only fits.

Table 3 Expected and observed yields in the Drell–Yan and VBF signal regions. The yields and uncertainties are presented after the background-only fit to the data in the Drell–Yan or VBF signal regions. The uncertainty in the total background estimate is smaller than the sum in quadrature of the individual background contributions due to anti-correlations between the estimated backgrounds from different sources

The post-fit m(WZ) distributions in the signal regions and their respective WZ-QCD and ZZ control regions are shown in Fig. 5 for the Drell–Yan selection and in Fig. 6 for the VBF selection. The bottom panels show that the observed mass distributions are in good agreement with the estimated post-fit background contributions in all signal and control regions.

The largest observed excess is in the VBF category at m(WZ) around 375 GeV, as shown in Fig. 6c. The local significances for VBF produced signals of a charged Higgs boson $H_5^{\pm }$ or an HVT $W'$ boson are 2.8 and 2.5 standard deviations, respectively. The respective global significances calculated using the look-elsewhere effect as in Ref. [104], and evaluated up to a mass of 1.2 TeV, are 1.6 and 1.7 standard deviations. In the Drell–Yan signal region the largest difference between the data and the SM background prediction is located around a mass of 1.1 TeV with a local significance of 1.2 standard deviations.

Fig. 5

Comparisons of the data and the expected background distributions of the WZ invariant mass in the a WZ-QCD, b the ZZ control regions and c the Drell–Yan signal region. The background predictions are obtained from a background-only simultaneous fit to the Drell–Yan signal region and its control regions. For illustration, the expected distributions from HVT $W'$ resonances (model A) with masses of 800 GeV and 1.4 TeV, with the latter normalized to four times its predicted cross-section, are shown in the signal region. The bottom panels show the ratios of the data to the post-fit background predictions. The uncertainty in the total background prediction, shown as a hashed area, combines statistical and systematic contributions

Fig. 6

Comparisons of the data and the expected background distributions of the WZ invariant mass in a the WZ-QCD VBF control region, b the ZZ VBF control region and c the VBF signal region. The background predictions are obtained from a background-only simultaneous fit to the VBF signal region and its two control regions. For illustration, the expected distributions from an $H_5^{\pm }$ GM model resonance ($\sin \theta _H= 0.5$) with a mass of 375 GeV and from an HVT $W'$ (model A) of mass 600 GeV are shown in the signal region, with the predicted cross sections scaled by 0.5 and 4, respectively. The bottom panels show the ratios of the data to the post-fit background predictions. The uncertainty in the total background prediction, shown as a hashed area, combines statistical and systematic contributions

8.3 Impact of systematic uncertainties

The effects of systematic uncertainties on the search are studied for hypothesized signals using a signal-strength parameter $\mu$, which is the ratio of the extracted cross-section to the injected hypothesized signal cross-section. For this study, the signal production cross-section is set to be equal to the expected median upper limits (Sect. 8.4). The expected relative uncertainties in the best-fit $\mu$ value after the maximum-likelihood fit are shown in Table 4 for two reference models and mass points: Drell–Yan production of a $W'$ boson in the HVT model with mass $m(W')$ = 1100 GeV, and VBF production of an $H^{\pm }_5$ in the GM model with mass $m(H^{\pm }_5)$ = 375 GeV. The individual sources of systematic uncertainty are combined into fewer background modelling and experimental categories. For signals with higher mass, the data statistical uncertainty is dominant. The uncertainties with the largest impact on the sensitivity of the searches are from the normalization of the irreducible backgrounds WZ-QCD and ZZ, from the theory modelling of the WZ background (in the table this includes QCD and EWK components), from the reducible background shape and normalization, and from the sizes of the MC samples. Uncertainties related to luminosity and pile-upalso play a relevant role in both signal regions. The jet uncertainties, such as those in the jet energy scale and resolution, naturally have a large impact in the VBF search.

Table 4 Dominant relative uncertainties in the best-fit signal-strength parameter ($\mu$) for a hypothetical HVT signal of mass $m(W')$ = 1 100 GeV in the Drell–Yan signal region and a GM signal of mass $m(H^{\pm }_5)$ = 375 GeV in the VBF signal region. For this study, the production cross-sections of the signals are set to the expected median upper limits at these two mass values. Uncertainties with smaller contributions are not included

8.4 Limits on the production of heavy resonances

Constraints on the production of heavy resonances are derived by repeating the test of the signal-plus-background hypothesis for different signal models. Upper limits on cross-sections times branching fraction to WZ are calculated using the asymptotic approximation [102].

For the HVT model search, Fig. 7 presents the observed and expected limits on $\sigma \times B(W' \rightarrow {WZ})$ at 95% CL as a function of the $W'$ mass for the HVT model in the Drell–Yan signal region. Masses below 2.4 TeV can be excluded for Model A and 2.5 TeV for Model B. For resonance masses above 2 TeV the exclusion limits become weaker due to the poorer acceptance at high mass (see Fig. 2). Regarding the VBF production mode, 95% CL limits on $\sigma \times B(W' \rightarrow {WZ})$ are shown in Fig. 8 for the benchmark model with $c_F=0$ and different values of $g_Vc_H$. Thus, they close a gap left by the limits obtained by searches in the Drell–Yan process as $c_F$ approaches zero.

Because of large mass mixing, which depends on the coupling $g_Vc_H$, between the SM gauge bosons and $V'$, the theory curves start at different values of V’ mass, close to the limit of validity of the HVT model. Masses below 340 GeV, 700 GeV, 945 GeV and 1145 GeV can be excluded for the HVT VBF model with $c_F = 0$ and $g_V c_H = 1.0$, 2.0, 3.0 and 4.0, respectively.

Fig. 7

Observed and expected 95% CL exclusion upper limits on $\sigma \times B(W' \rightarrow {WZ})$ for the Drell–Yan production of a $W'$ boson in the HVT model as a function of its mass. The LO theory predictions for HVT Model A with $g_V=1$ and Model B with $g_V=3$ are also shown

For the $H^{\pm }_5$ GM search, observed and expected exclusion limits at 95% CL on $\sigma \times B(H^{\pm }_5 \rightarrow {WZ})$ and on the mixing parameter $\sin \theta _H$ are shown in Fig. 9. The latter are about 35% stronger than in the previous publication [24]. They are comparable to the CMS results [36], based on the combination of searches for singly and doubly charged members of the $H_5$ fiveplet. The shaded regions show the parameter space for which the $H^{\pm }_5$ width exceeds 5% and 10% of $m_{H^{\pm }_5}$. The intrinsic width of the scalar resonance is narrower than the detector resolution in the mass region explored.

Fig. 8

Observed and expected 95% CL upper limits on $\sigma \times B(W' \rightarrow {WZ})$ for the VBF production of a $W'$ boson in the HVT benchmark with parameter $c_F$ = 0, as a function of its mass. The LO theory predictions for HVT VBF benchmark with different values of the coupling parameters $g_V$ and $c_H$ are also shown

Fig. 9

Observed and expected 95% CL upper limits a on $\sigma \times B(H^{\pm }_5 \rightarrow {WZ})$ and b on the parameter $\textrm{sin} \theta _H$ of the GM model as a function of $m_{H^{\pm }_5}$. The shaded region shows where the theoretical intrinsic width of the resonance would be larger than 5% or 10% of the mass

As a test of the asymptotic approximation used in the statistical analysis, limits are also computed with ensembles of pseudo-experiments in all signal and control regions. The cross-section upper limits obtained in that case agree in all cases for masses below 500 GeV. At higher masses, where event yields become smaller, the discrepancy between the two methods become larger, but they remain within 6–10%.

9 Conclusions

A search was performed for resonant WZ production in fully leptonic final states (electrons and muons) using 139 fb$^{-1}$ of $\sqrt{s} = 13~\textrm{TeV}$ pp collision data collected by the ATLAS experiment at the LHC. Two different production processes are considered, Drell–Yan and vector-boson fusion.

The data in the Drell–Yan selection are found to be consistent with Standard Model predictions. The results are used to derive upper limits at 95% confidence level on the cross-section times branching ratio for heavy vector triplet production in benchmark Model A (Model B) with coupling constant $g_V=1$ ($g_V=3$) as a function of the resonance mass, with no evidence of heavy vector resonance production for masses below 2.4 (2.5) TeV.

In the case of the VBF production process, limits on the production cross-section times branching ratio to WZ of a hypothetical resonance are obtained as a function of the mass for a heavy vector triplet or for a charged member of the fiveplet scalar in the Georgi–Machacek model. Values of the parameter $\sin \theta _H > 0.3$ are excluded for masses between 200 GeV and 1 TeV. The results show a local excess of events over the Standard Model expectations at a resonance mass of around 375 GeV. The local significances for signals of a heavy vector $W'$ boson or a $H_5^{\pm }$ are 2.5 and 2.8 standard deviations respectively. The respective global significances calculated considering the look-elsewhere effect are 1.7 and 1.6 standard deviations respectively. With no evidence of heavy $W'$ vector-resonance production, limits on the production times branching ratio for the heavy vector triplet VBF production process have been obtained as a function of mass.

Appendix

For the vector-boson fusion analysis a cut-based selection was also developed, to facilitate reinterpretations of our results outside the ATLAS Collaboration and to serve in parallel to confirm the stability of the results obtained with the ANN.

The VBF cut-based signal region selection is built on top of the baseline WZ selection described in Sect. 5.1 by requiring events to have at least two VBF jets with $m_{\textit{jj}}>$ 500 GeV and rapidity separation $|\Delta y_{\textit{jj}}|>$ 3.5. The combined detector acceptance and signal selection efficiency $\mathcal {A\times \epsilon }$ of the cut-based VBF selection as a function of the mass of the VBF $H^{\pm }_{5}$ and of the HVT $W'$ boson, relative to the generated signal events, is shown in Fig. 10.

Fig. 10

The acceptance times efficiency of VBF $H^{\pm }_{5}$ and HVT $W'$ selection using a cut-based VBF selection is shown at different mass points for the individual channels $\mu ^{+}\mu ^{-}\mu ^{\pm }, \mu ^{+}\mu ^{-} e^{\pm }, e^{+}e^{-}\mu ^{\pm }, e^{+}e^{-}e^{\pm }$ and the sum of all channels. The error includes statistical and experimental systematic uncertainties

Following the same strategy as the nominal analysis, two control regions are defined so as to constrain the WZ-QCD and ZZ background. The WZ-QCD VBF cut-based control region is defined by inverting the $|\Delta y_{\textit{jj}}|$ requirement, and the nominal ZZ VBF control region is used. Figure 11 shows comparisons of the data and the expected background distributions. The background predictions are obtained from a background-only simultaneous fit to the VBF cut-based signal region and the WZ-QCD and ZZ VBF control regions.

Fig. 11

Comparisons of the observed data and the expected background distributions of the WZ invariant mass using the cut based VBF selection are shown for a WZ-QCD VBF control region, b ZZ VBF control region and c the VBF signal region. The background predictions are obtained from a background-only simultaneous fit to the VBF cut-based signal region and the WZ-QCD and ZZ VBF control regions. For illustration, the expected distributions from an $H_5^{\pm }$ GM model resonance ($\sin \theta _H = 0.5)$ with mass of 375 GeV and from an HVT $W'$ (model A) of mass 600 GeV are shown in the signal region, with the latter predicted cross sections scaled by 10. The bottom panel show the ratio of the observed data to the background predictions. The uncertainty in the total background prediction, shown as a hashed area, combines statistical and systematic contributions

Constraints on the production of heavy resonances are derived by repeating the fit to the signal-plus-background hypothesis for different signal models using the cut-based analysis. For the HVT model search, Fig. 12 presents the observed and expected limits on $\sigma \times B(W' \rightarrow {WZ})$ at 95% CL as a function of the $W'$ mass. Masses below 370 $\textrm{GeV}$, 590 $\textrm{GeV}$, 895 $\textrm{GeV}$ and 1100 $\textrm{GeV}$ for HVT VBF production can be excluded for the HVT VBF model with $c_F = 0$ and $g_V c_H = 1.0$, 2.0, 3.0 and 4.0, respectively. The sensitivity of the ANN-based analysis is better than that of the cut-based analysis thanks to the utilization of the neural network that better exploits the difference between signals and backgrounds. In the cut-based analysis the relative contribution of background is more important in the signal region, leading to overall higher expected and observed limits. In the region around 375 GeV in particular due to statistics fluctuations in the data both methods of analysis yield consistent observed limits.

Fig. 12

Using the cut-based VBF selection, the observed and expected 95% CL upper limits on $\sigma \times B(W' \rightarrow {WZ})$ for the VBF production of a $W'$ boson in the HVT with parameter $c_F$ = 0, as a function of its mass. The LO theory predictions for HVT VBF model with different values of the coupling parameters $g_V$ and $c_H$ are also shown

Fig. 13

Using the cut-based VBF selection, the observed and expected 95% CL upper limits on a the $\sigma \times B(H^{\pm }_5 \rightarrow {WZ})$ and b the parameter $\sin \theta _H$ of the GM Model as a function of $m_{H^{\pm }_5}$. The shaded regions show where the theoretical intrinsic width of the resonance would be larger than 5% and 10% of the mass

For the $H^{\pm }_5$ GM search, observed and expected exclusion limits at 95% CL on $\sigma \times B(H^{\pm }_5 \rightarrow {WZ})$ and on the mixing parameter $\sin \theta _H$ are shown in Fig. 13. The intrinsic width of the scalar resonance, for $\sin \theta _H$= 0.5, is narrower than the detector resolution in the mass region explored. The shaded regions show the parameter space where the $H^{\pm }_5$ width exceeds 5% and 10% of $m_{H^{\pm }_5}$. The expected limits extracted using the cut-based analysis are, for both models, between 30 and 50% less stringent than the ones extracted using the ANN signal region selection.