Skip to main content
SpringerLink
Log in

Strong Gravitational Lensing Parameter Estimation with Vision Transformer

  • Conference paper
  • First Online:
Book cover Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13801))

Included in the following conference series:

  • 1351 Accesses

Abstract

Quantifying the parameters and corresponding uncertainties of hundreds of strongly lensed quasar systems holds the key to resolving one of the most important scientific questions: the Hubble constant (\(H_{0}\)) tension. The commonly used Markov chain Monte Carlo (MCMC) method has been too time-consuming to achieve this goal, yet recent work has shown that convolution neural networks (CNNs) can be an alternative with seven orders of magnitude improvement in speed. With 31,200 simulated strongly lensed quasar images, we explore the usage of Vision Transformer (ViT) for simulated strong gravitational lensing for the first time. We show that ViT could reach competitive results compared with CNNs, and is specifically good at some lensing parameters, including the most important mass-related parameters such as the center of lens \(\theta _{1}\) and \(\theta _{2}\), the ellipticities \(e_1\) and \(e_2\), and the radial power-law slope \(\gamma '\). With this promising preliminary result, we believe the ViT (or attention-based) network architecture can be an important tool for strong lensing science for the next generation of surveys. The open source of our code and data is in https://github.com/kuanweih/strong_lensing_vit_resnet.

K.-W. Huang, G.C.-F. Chen and Y.-Y. Lin—Equal contribution

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Perlmutter, S., et al.: Measurements of omega and lambda from 42 high-redshift supernovae. Astrophys. J. 517, 565–586 (1999)

    Article  MATH  Google Scholar 

  2. Riess, A.G., et al.: Observational evidence from supernovae for an accelerating universe and a cosmological constant. Astron. J. 116, 1009–1038 (1998)

    Article  Google Scholar 

  3. Hinshaw, G., et al.: Nine-year Wilkinson microwave anisotropy probe (WMAP) observations: cosmological parameter results. Astrophys. J. Suppl. Ser. 208, 19 (2013)

    Article  Google Scholar 

  4. Riess, A.G., et al.: Cosmic distances calibrated to 1% precision with Gaia EDR3 parallaxes and Hubble space telescope photometry of 75 Milky Way Cepheids confirm tension with \(\Lambda \)CDM. Astrophys. J. Lett. 908(1), L6 (2021)

    Article  MathSciNet  Google Scholar 

  5. Planck Collaboration, N., et al.: Planck 2018 results. VI. Cosmological parameters. Astron. Astrophys. 641, A6 (2020)

    Article  Google Scholar 

  6. Freedman, W.L., et al.: Calibration of the tip of the red giant branch. Astrophys. J. 891(1), 57 (2020)

    Article  Google Scholar 

  7. Wong, K.C., et al.: H0LiCOW – XIII. A 2.4 per cent measurement of H\(_{0}\) from lensed quasars: 5.3\(\sigma \) tension between early- and late-Universe probes. Monthly Notices R. Astron. Soc. 498(1), 1420–1439 (2020)

    Google Scholar 

  8. Treu, T., Marshall, P.J.: Time delay cosmography. Astron. Astrophys. Rev. 24, 11 (2016)

    Article  Google Scholar 

  9. Suyu, S.H., Chang, T.-C., Courbin, F., Okumura, T.: Cosmological distance indicators. Space Sci. Rev. 214(5), 91 (2018)

    Article  Google Scholar 

  10. Refsdal, S.: On the possibility of determining Hubble’s parameter and the masses of galaxies from the gravitational lens effect. Mon. Not. R. Astron. Soc. 128, 307 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  11. Wong, K.C., et al.: H0LiCOW - IV. Lens mass model of HE 0435–1223 and blind measurement of its time-delay distance for cosmology. Mon. Not. R. Astron. Soc. 465, 4895–4913 (2017)

    Google Scholar 

  12. S. Birrer, et al. H0LiCOW - IX. Cosmographic analysis of the doubly imaged quasar SDSS 1206+4332 and a new measurement of the Hubble constant. Mon. Not. R. Astron. Soc. 484, 4726–4753 (2019)

    Google Scholar 

  13. Rusu, C.E., et al.: H0LiCOW XII. Lens mass model of WFI2033-4723 and blind measurement of its time-delay distance and H\(_{0}\). Mon. Not. R. Astron. Soc. 498(1), 1440–1468 (2020)

    Google Scholar 

  14. Chen, G.C.-F., et al.: Constraining the microlensing effect on time delays with a new time-delay prediction model in H\(_{0}\) measurements. Mon. Not. R. Astron. Soc. 481(1), 1115–1125 (2018)

    Google Scholar 

  15. Chen, G.C.F., et al.: SHARP - VIII. J 0924+0219 lens mass distribution and time-delay prediction through adaptive-optics imaging. Mon. Not. R. Astron. Soc. 513, 2349–2359 (2022)

    Google Scholar 

  16. Shajib, A.J., et al.: STRIDES: a 3.9 per cent measurement of the Hubble constant from the strong lens system DES J0408–5354. Mon. Not. R. Astron. Soc. 494(4), 6072–6102 (2020)

    Google Scholar 

  17. LSST Science Collaboration, et al.: LSST Science Book, Version 2.0. arXiv e-prints, page arXiv:0912.0201, December 2009

  18. Oguri, M., Marshall, P.J.: Gravitationally lensed quasars and supernovae in future wide-field optical imaging surveys. Mon. Not. R. Astron. Soc. 405, 2579–2593 (2010)

    Google Scholar 

  19. Hezaveh, Y.D., Levasseur, L.P., Marshall, P.J.: Fast automated analysis of strong gravitational lenses with convolutional neural networks. Nature. 548(7669), 555–557 (2017)

    Google Scholar 

  20. Levasseur, L.P., Hezaveh, Y.D., Wechsler, R.H.: Uncertainties in parameters estimated with neural networks: application to strong gravitational lensing. Astrophys. J. Lett. 850(1), L7 (2017)

    Google Scholar 

  21. Brehmer, J., Mishra-Sharma, S., Hermans, J., Louppe, G., Cranmer, K.: Mining for dark matter substructure: inferring sub halo population properties from strong lenses with machine learning. Astrophys. J. 886(1), 49 (2019)

    Article  Google Scholar 

  22. Wagner-Carena, S., et al.: Hierarchical inference with Bayesian neural networks: an application to strong gravitational lensing. Astrophys. J. 909(2), 187 (2021)

    Google Scholar 

  23. Lin, J.Y.-Y., Yu, H., Morningstar, W., Peng, J., Holder, G.: Hunting for dark matter Subhalos in strong gravitational lensing with neural networks. In: 34th Conference on Neural Information Processing Systems, October 2020

    Google Scholar 

  24. Park, J.W., et al.: Large-scale gravitational lens modeling with Bayesian neural networks for accurate and precise inference of the Hubble constant. Astrophys. J. 910(1), 39 (2021)

    Article  MathSciNet  Google Scholar 

  25. Morgan, R., Nord, B., Birrer, S., Lin, J.Y.-Y., Poh, J.: Deeplenstronomy: a dataset simulation package for strong gravitational lensing. J. Open Source Softw. 6(58), 2854 (2021)

    Article  Google Scholar 

  26. Morningstar, W.R., et al.: Analyzing interferometric observations of strong gravitational lenses with recurrent and convolutional neural networks. arXiv preprint arXiv:1808.00011 (2018)

  27. Coogan, A., Karchev, K., Weniger, C.: Targeted likelihood-free inference of dark matter substructure in strongly-lensed galaxies. In 34th Conference on Neural Information Processing Systems, October 2020

    Google Scholar 

  28. Ostdiek, B., Rivero, A.D., Dvorkin, C.: Extracting the subhalo mass function from strong lens images with image segmentation. Astrophys. J. 927(1), 3 (2022)

    Google Scholar 

  29. Ostdiek, B., Rivero, A.D., Dvorkin, C.: Image segmentation for analyzing galaxy-galaxy strong lensing systems. Astron. Astrophys. 657, L14 (2022)

    Article  Google Scholar 

  30. Thuruthipilly, H., Zadrozny, A., Pollo, A.: Finding strong gravitational lenses through self-attention. arXiv preprint arXiv:2110.09202 (2021)

  31. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  32. Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, 3–7 May 2021. OpenReview.net, 2021

    Google Scholar 

  33. Paul, S., Chen, P.-Y.: Vision transformers are robust learners. In: AAAI (2022)

    Google Scholar 

  34. Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., Dosovitskiy, A.: Do vision transformers see like convolutional neural networks? In: Beygelzimer, A., Dauphin, Y., Liang, P., Wortman Vaughan, J. (eds.) Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  35. Birrer, S., Amara, A.: lenstronomy: multi-purpose gravitational lens modelling software package. Phys. Dark Univ. 22, 189–201 (2018)

    Article  Google Scholar 

  36. Birrer, S., et al.: Lenstronomy ii: a gravitational lensing software ecosystem. J. Open Sourc. Softw. 6(62), 3283 (2021)

    Google Scholar 

  37. Suyu, S.H., et al.: Two accurate time-delay distances from strong lensing: implications for cosmology. Astrophys. J. 766, 70 (2013)

    Article  Google Scholar 

  38. Barkana, R.: Fast calculation of a family of elliptical mass gravitational lens models. Astrophys. J. 502, 531 (1998)

    Article  Google Scholar 

  39. Sérsic, J.L.: Atlas de galaxias Australes. Observatorio Astronomico, Cordoba, Argentina (1968)

    Google Scholar 

  40. Krist, J.E., Hook, R.N.: NICMOS PSF variations and tiny Tim simulations. In: Casertano, S., Jedrzejewski, R., Keyes, T., Stevens, M. (eds.) The 1997 HST Calibration Workshop with a New Generation of Instruments, p. 192, January 1997

    Google Scholar 

  41. Chen, G.C.-F., et al.: SHARP - III. First use of adaptive-optics imaging to constrain cosmology with gravitational lens time delays. Mon. Not. R. Astron. Soc. 462, 3457–3475 (2016)

    Google Scholar 

  42. Chen, G.C.-F., et al.: A SHARP view of H0LiCOW: H\(_{0}\) from three time-delay gravitational lens systems with adaptive optics imaging. Mon. Not. R. Astron. Soc. 490(2), 1743–1773 (2019)

    Google Scholar 

  43. Geoff C.-F. Chen, Treu, T., Fassnacht, C.D., Ragland, S., Schmidt, T., Suyu, S.H.: Point spread function reconstruction of adaptive-optics imaging: meeting the astrometric requirements for time-delay cosmography. Mon. Not. R. Astron. Soc. 508(1), 755–761 (2021)

    Google Scholar 

  44. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.-F.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  45. Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45. Association for Computational Linguistics, October 2020

    Google Scholar 

  46. He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv e-prints, arXiv:1512.03385, December 2015

  47. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R., (eds.), Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates Inc (2019)

    Google Scholar 

  48. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Balcan, M.F., Weinberger, K.Q. (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp. 1050–1059. PMLR, New York, New York, USA, 20–22 June 2016

    Google Scholar 

  49. Kendall, A., Gal, Y.: What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv e-prints. arXiv:1703.04977, March 2017

  50. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv e-prints. arXiv:1412.6980, December 2014

  51. Abdalla, E., et al.: Cosmology intertwined: a review of the particle physics, astrophysics, and cosmology associated with the cosmological tensions and anomalies. J. High Energy Astrophys. 34, 49–211 (2022)

    Google Scholar 

  52. Suyu, S.H., et al.: The Hubble constant and new discoveries in cosmology. ArXiv e-prints. arxiv:1202.4459, February 2012

  53. Falco, E.E., Gorenstein, M.V., Shapiro, I.I.: On model-dependent bounds on H(0) from gravitational images Application of Q0957 + 561A.B. Astrophys. J. Lett. 289, L1–L4 (1985)

    Article  Google Scholar 

  54. Gorenstein, M.V., Falco, E.E., Shapiro, I.I.: Degeneracies in parameter estimates for models of gravitational lens systems. Astrophys. J. 327, 693 (1988)

    Article  Google Scholar 

  55. Schneider, P., Sluse, D.: Mass-sheet degeneracy, power-law models and external convergence: impact on the determination of the Hubble constant from gravitational lensing. Astron. Astrophys. 559, A37 (2013)

    Article  Google Scholar 

  56. Xu, D., et al.: Lens galaxies in the Illustris simulation: power-law models and the bias of the Hubble constant from time delays. Mon. Not. R. Astron. Soc. 456, 739–755 (2016)

    Article  Google Scholar 

  57. Gomer, M., Williams, L.L.R.: Galaxy-lens determination of H\(_{0}\): constraining density slope in the context of the mass sheet degeneracy. J. Cosmol. Astropart. Phys. 2020(11), 045 (2020)

    Article  Google Scholar 

  58. Kochanek, C.S.: Over constrained gravitational lens models and the Hubble constant. Mon. Not. R. Astron. Soc. 493(2), 1725–1735 (2020)

    Article  Google Scholar 

  59. Blum, K., Castorina, E., Simonović, M.: Could quasar lensing time delays hint to a core component in Halos, instead of H\(_{0}\) tension? Astrophys. J. Lett. 892(2), L27 (2020)

    Article  Google Scholar 

  60. Millon, M., et al.: TDCOSMO. I. An exploration of systematic uncertainties in the inference of H\(_{0}\) from time-delay cosmography. Astron. Astrophys. 639, A101 (2020)

    Google Scholar 

  61. Ding, X., et al.: Time delay lens modelling challenge. Mon. Not. R. Astron. Soc. 503(1), 1096–1123 (2021)

    Article  MathSciNet  Google Scholar 

  62. Birrer, S., et al.: TDCOSMO. IV. Hierarchical time-delay cosmography – joint inference of the Hubble constant and galaxy density profiles. Astron. Astrophys. 643, A165 (2020)

    Google Scholar 

  63. Chen, G.C.-F., Fassnacht, C.D., Suyu, S.H., Yıldırım, A., Komatsu, E., Bernal, J.L.: TDCOSMO. VI. Distance measurements in time-delay cosmography under the mass-sheet transformation. Astron. Astrophys. 652, A7 (2021)

    Google Scholar 

  64. Tagore, A.S., et al.: Reducing biases on H\(_{0}\) measurements using strong lensing and galaxy dynamics: results from the EAGLE simulation. Mon. Not. R. Astron. Soc. 474(3), 3403–3422 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, USA

    Kuan-Wei Huang

  2. University of California, Los Angeles, USA

    Geoff Chih-Fan Chen

  3. Ohio State University, Columbus, USA

    Po-Wen Chang

  4. University of Kentucky, Lexington, USA

    Sheng-Chieh Lin

  5. Chalmers University of Technology, Gothenburg, Sweden

    ChiaJung Hsu

  6. Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE

    Vishal Thengane

  7. University of Illinois at Urbana-Champaign, Champaign, USA

    Joshua Yao-Yu Lin

Authors
  1. Kuan-Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoff Chih-Fan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Po-Wen Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sheng-Chieh Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. ChiaJung Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vishal Thengane
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joshua Yao-Yu Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joshua Yao-Yu Lin .

Editor information

Editors and Affiliations

  1. IBM Research AI and MIT-IBM Watson AI Lab, Haifa, Israel

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, KW. et al. (2023). Strong Gravitational Lensing Parameter Estimation with Vision Transformer. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25056-9_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25055-2

  • Online ISBN: 978-3-031-25056-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation