As part of the research on thermal noise reduction in gravitational-wave detectors, we experimentally demonstrate the conversion of a fundamental ${\mathrm{TEM}}_{00}$ laser mode at 1064 nm to higher-order Hermite-Gaussian modes (HG) of arbitrary order via a commercially available liquid crystal spatial light modulator. We particularly studied the ${\mathrm{HG}}_{5,5}/{\mathrm{HG}}_{10,10}/{\mathrm{HG}}_{15,15}$ modes. A two-mirror plano-spherical cavity filters the higher-order modes spatially. We analyze the cleaned modes via a three-mirror diagnosis cavity and measure a mode purity of $96/93/78\%$ and a conversion efficiency of $6.6\%/3.7\%/1.7\%$, respectively. A full set of simulations and mathematical proofs are also presented which shows that (i) Hermite-Gauss modes resonate in a two-mirror cavity provided mirrors are properly angled with respect to the impinging mode, and (ii) Hermite-Gauss modes resonate in triangular cavities. Hence, higher-order Hermite-Gauss modes are compatible with ground-based gravitational-wave detectors’ architecture and can be employed for the mitigation of mirror thermal noise for the third generation Einstein Telescope or Cosmic Explorer.

• Received 4 February 2019
• Revised 7 March 2020
• Accepted 7 January 2021

DOI:https://doi.org/10.1103/PhysRevD.103.042008