Skip to main content
SpringerLink
Log in

Beyond Visible Light: User and Societal Impacts of Egocentric Multispectral Vision

  • Conference paper
  • First Online:
Virtual, Augmented and Mixed Reality (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12770))

Included in the following conference series:

  • 2958 Accesses

Abstract

Multi-spectral imagery is becoming popular for a wide range of application fields from agriculture to healthcare, mainly stemming from advances in consumer sensor and display technologies.Modern augmented reality (AR) head-mounted displays already combine a multitude of sensors and are well-suited for integration with additional sensors, such as cameras capturing information from different parts of the electromagnetic spectrum. In this paper, we describe a novel multi-spectral vision prototype based on the Microsoft HoloLens 1, which we extended with two thermal infrared (IR) cameras and two ultraviolet (UV) cameras. We performed an exploratory experiment, in which participants wore the prototype for an extended period of time and assessed its potential to augment our daily activities. Our report covers a discussion of qualitative insights on personal and societal uses of such novel multi-spectral vision systems, including their applicability for use during the COVID-19 pandemic.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ai, J., Jin, L., Zhang, Y., Tian, Z., Peng, C., Duan, W.: Detecting partial discharge of polluted insulators based on ultraviolet imaging. In: IEEE 11th International Conference on the Properties and Applications of Dielectric Materials (ICPADM), pp. 456–459. IEEE (2015)

    Google Scholar 

  2. Akula, A., Ghosh, R., Sardana, H.K.: Thermal imaging and its application in defence systems. Optics: Phenomena, Materials, Devices, and Characterization, pp. 333–335 (2011)

    Google Scholar 

  3. Bae, W., Kim, Y., Ahn, S.: IR-band conversion of target and background using surface temperature estimation and error compensation for military IR sensor simulation. Sensors 19(11), 2455 (2019)

    Article  Google Scholar 

  4. Balaras, C.A., Argiriou, A.A.: Infrared thermography for building diagnostics. Energy Build. 34(2), 171–183 (2002)

    Article  Google Scholar 

  5. Becker, L.: Influence of IR sensor technology on the military and civil defense. In: Quantum Sensing and Nanophotonic Devices III, vol. 6127, p. 61270S (2006)

    Google Scholar 

  6. Bendada, A., Sfarra, S., Ibarra, C., Akhloufi, M., Pradere, C., Maldague, X.: Subsurface imaging for panel paintings inspection: a comparative study of the ultraviolet, the visible, the infrared and the terahertz spectra. Opto-Electron. Rev. 23(1), 90–101 (2015)

    Article  Google Scholar 

  7. Bennett, M.V., Matthews, I.: Life-saving uncooled IR camera for use in firefighting applications, vol. 2744 (1996)

    Google Scholar 

  8. Bento, N.A.F., Silva, J.S., Dias, J.B.: Detection of camouflaged people. International Journal of Sensor Networks and Data Communications, vol. 5, no. 3 (2016)

    Google Scholar 

  9. Bluth, G.J.S., Shannon, J.M., Watson, I.M., Prata, A.J., Realmuto, V.J.: Development of an ultra-violet digital camera for volcanic so2 imaging. J. Volcanol. Geoth. Res. 161(1–2), 47–56 (2007)

    Article  Google Scholar 

  10. Boushel, R., Piantadosi, C.A.: Near-infrared spectroscopy for monitoring muscle oxygenation. Acta Physiol. Scand. 168(4), 615–622 (2000)

    Article  Google Scholar 

  11. Cardone, D., Merla, A.: New frontiers for applications of thermal infrared imaging devices: computational psychopshysiology in the neurosciences. Sensors 17(5), 1042 (2017)

    Google Scholar 

  12. Centers for Disease Control and Prvention (CDC): High blood pressure fact sheet, June 2016. https://www.cdc.gov/dhdsp/data_statistics/fact_sheets/fs_bloodpressure.htm

  13. Chen, Y.R., Chao, K., Kim, M.S.: Machine vision technology for agricultural applications. Comput. Electron. Agric. 36(2–3), 173–191 (2002)

    Article  Google Scholar 

  14. Cho, Y.: Mobile Thermography-based Physiological Computing for Automatic Recognition of a Person’s Mental Stress. Ph.D. thesis, UCL Interaction Centre (2019)

    Google Scholar 

  15. Erickson, A., Kim, K., Bruder, G., Welch, G.: Exploring the limitations of environment lighting on optical see-through head-mounted displays. In: Proceedings of ACM Conference on Spatial User Interfaces (SUI), pp. 1–8 (2020)

    Google Scholar 

  16. Erickson, A., Schubert, R., Kim, K., Bruder, G., Welch, G.: Is it cold in here or is it just me? analysis of augmented reality temperature visualization for computer-mediated thermoception. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR) (2019)

    Google Scholar 

  17. Fulton, J.E., Jr.: Utilizing the ultraviolet (uv detect) camera to enhance the appearance of photodamage and other skin conditions. Dermatol. Surg. 23(3), 163–169 (1997)

    Article  Google Scholar 

  18. Gade, R., Moeslund, T.B.: Thermal cameras and applications: a survey. Mach. Vis. Appl. 25(1), 245–262 (2014)

    Article  Google Scholar 

  19. Glushakow, J.M.: Social facilitation: salience and mediated, anticipatory, and residual presence. Ph.D. thesis, Rutgers University-Graduate School-New Brunswick (2011)

    Google Scholar 

  20. He, S., Wang, S., Lan, W., Fu, H., Ji, Q.: Facial expression recognition using deep boltzmann machine from thermal infrared images. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, pp. 239–244 (2013)

    Google Scholar 

  21. Hermanson, A.S., Bush, M.A., Miller, R.G., Bush, P.J.: Ultraviolet illumination as an adjunctive aid in dental inspection. J. Forensic Sci. 53(2), 408–411 (2008)

    Article  Google Scholar 

  22. Hu, M., et al.: Combination of near-infrared and thermal imaging techniques for the remote and simultaneous measurements of breathing and heart rates under sleep situation. Plos One 13(1), 1–14 (2018)

    Google Scholar 

  23. Ishimwe, R., Abutaleb, K., Ahmed, F.: Applications of thermal imaging in agriculture - a review. Adv. Remote Sens. 03(03), 128–140 (2014)

    Article  Google Scholar 

  24. Jiang, L.J., et al.: A perspective on medical infrared imaging. J. Med. Eng. Technol. 29(6), 257–267 (2005)

    Article  Google Scholar 

  25. Jones, B.F.: A reappraisal of the use of infrared thermal image analysis in medicine. IEEE Trans. Med. Imaging 17(6), 1019–1027 (1998)

    Article  Google Scholar 

  26. Kruse, P.W.: Uncooled Thermal Imaging Arrays, Systems, and Applications. SPIE Press, Bellingham (2001)

    Google Scholar 

  27. Lagüela, S., Armesto, J., Arias, P., Herráez, J.: Automation of thermographic 3D modelling through image fusion and image matching techniques. Autom. Constr. 27, 24–31 (2012)

    Article  Google Scholar 

  28. Lamb, D.W.: The use of qualitative airborne multispectral imaging for managing agricultural crops - a case study in south-eastern Australia. Aust. J. Exp. Agric. 40(5), 725 (2000)

    Article  Google Scholar 

  29. Lo, C.P., Quattrochi, D.A., Luvall, J.C.: Application of high-resolution thermal infrared remote sensing and GIS to assess the urban heat island effect. Int. J. Remote Sens. 18(2), 287–304 (1997)

    Article  Google Scholar 

  30. Mason, R.S., Sequeira, V.B., Gordon-Thomson, C.: Vitamin D: the light side of sunshine. Eur. J. Clin. Nutr. 65(9), 986 (2011)

    Article  Google Scholar 

  31. McGonigle, A.J.S., et al.: Ultraviolet imaging of volcanic plumes: a new paradigm in volcanology. Geosciences 7(3), 68 (2017)

    Article  Google Scholar 

  32. Morlier, A., Siebert, M., Kunze, I., Mathiak, G., Köntges, M.: Detecting photovoltaic module failures in the field during daytime with ultraviolet fluorescence module inspection. IEEE J. Photovoltaics 7(6), 1710–1716 (2017)

    Article  Google Scholar 

  33. Muehleisen, W., et al.: Outdoor detection and visualization of hailstorm damages of photovoltaic plants. Renew. Energy 118, 138–145 (2018)

    Article  Google Scholar 

  34. Munnerley, D., Bacon, M., Wilson, A., Steele, J., Hedberg, J., Fitzgerald, R.: Confronting an augmented reality (2012)

    Google Scholar 

  35. Narayanan, D.L., Saladi, R.N., Fox, J.L.: Review: ultraviolet radiation and skin cancer. Int. J. Dermatol. 49(9), 978–986 (2010)

    Article  Google Scholar 

  36. Ochoa, D., et al.: Hyperspectral imaging system for disease scanning on banana plants. In: Sensing for Agriculture and Food Quality and Safety VIII, vol. 9864, p. 98640M. International Society for Optics and Photonics (2016)

    Google Scholar 

  37. Orlosky, J., et al.: Vismerge: light adaptive vision augmentation via spectral and temporal fusion of non-visible light. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 22–31 (2017)

    Google Scholar 

  38. Panov, V., Borisova-Papancheva, T.: Application of ultraviolet light (uv) in dental medicine. Med. Inform. 2, 194–200 (2015)

    Google Scholar 

  39. Pavlidis, I., Levine, J., Baukol, P.: Thermal image analysis for anxiety detection. In: International Conference on Image Processing, vol. 2, pp. 315–318. IEEE (2001)

    Google Scholar 

  40. Puri, V., Nayyar, A., Raja, L.: Agriculture drones: a modern breakthrough in precision agriculture. J. Stat. Manag. Syst. 20(4), 507–518 (2017)

    Google Scholar 

  41. Reed, N.G.: The history of ultraviolet germicidal irradiation for air disinfection. Publ. Health Rep. 125(1), 15–27 (2010)

    Article  MathSciNet  Google Scholar 

  42. Roberts, J.E.: Ultraviolet radiation as a risk factor for cataract and macular degeneration. Eye Contact Lens 37(4), 246–9 (2011)

    Article  Google Scholar 

  43. Rudol, P., Doherty, P.: Human body detection and geolocalization for UAV search and rescue missions using color and thermal imagery. In: IEEE Aerospace Conference, pp. 1–8. IEEE (2008)

    Google Scholar 

  44. Sankaran, S., Mishra, A., Ehsani, R., Davis, C.: A review of advanced techniques for detecting plant diseases. Comput. Electron. Agric. 72(1), 1–13 (2010)

    Article  Google Scholar 

  45. Sosnowski, T., Madura, H., Bieszczad, G., Kastek, M.: Thermal camera system integrated into firefighter helmet. Measurement Automation Monitoring, vol. 63 (2017)

    Google Scholar 

  46. Stockton, G.R.: The use of thermal imaging technology. Technical report, Stockton Infrared Thermographic Services, Inc. (2015)

    Google Scholar 

  47. Torresan, H., Turgeon, B., Ibarra-Castanedo, C., Hebert, P., Maldague, X.P.: Advanced surveillance systems: combining video and thermal imagery for pedestrian detection. In: Thermosense XXVI, vol. 5405, p. 506 (2004)

    Google Scholar 

  48. Van Eck, W., Kolstee, Y.: The augmented painting: playful interaction with multi-spectral images. In: 2012 IEEE International Symposium on Mixed and Augmented Reality-arts, Media, and Humanities (ISMAR-AMH), pp. 65–69. IEEE (2012)

    Google Scholar 

  49. Vogel, A., et al.: Using noninvasive multispectral imaging to quantitatively assess tissue vasculature. J. Biomed. Opt. 12(5), 051604 (2007)

    Google Scholar 

  50. Wilkes, T., et al.: Ultraviolet imaging with low cost smartphone sensors: development and application of a raspberry Pi-based UV camera. Sensors 16(10), 1649 (2016)

    Article  MathSciNet  Google Scholar 

  51. Wilkes, T., Pering, T., McGonigle, A., Tamburello, G., Willmott, J.: A low-cost smartphone sensor-based UV camera for volcanic so2 emission measurements. Remote Sens. 9(1), 27 (2017)

    Article  Google Scholar 

  52. Wilson, A.D., Zhang, Z., Chou, P.A., Fishman, N.S., Gillett, D.M., LLC, H.B.M.T.L.: Providing a tele-immersive experience using a mirror metaphor. US patent no. us9641805b2 (2017)

    Google Scholar 

  53. Wright, F., Weller, R.B.: Risks and benefits of UV radiation in older people: more of a friend than a foe? Maturitas 81(4), 425–431 (2015)

    Article  Google Scholar 

  54. Zhang, X., Xu, W., Huang, M.C., Amini, N., Ren, F.: See UV on your skin: an ultraviolet sensing and visualization system. In: Proceedings of the 8th International Conference on Body Area Networks, pp. 22–28. BodyNets 2013, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium (2013)

    Google Scholar 

  55. Zhou, D., Dillon, M., Kwon, E.: Tracking-based deer vehicle collision detection using thermal imaging. In: IEEE International Conference on Robotics and Biomimetics, pp. 688–693. IEEE (2009)

    Google Scholar 

  56. Zhu, Z., Li, W., Wolberg, G.: Integrating LDV audio and IR video for remote multimodal surveillance. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, p. 10 (2005)

    Google Scholar 

Download references

Acknowledgments

This material includes work supported in part by the Office of Naval Research under Award Number N00014-17-1-2927 (Dr. Peter Squire, Code 34) and the Advent Health Endowed Chair in Healthcare Simulation (Prof. Welch).

Author information

Authors and Affiliations

  1. University of Central Florida, Orlando, FL, 32816, USA

    Austin Erickson, Kangsoo Kim, Gerd Bruder & Gregory F. Welch

Authors
  1. Austin Erickson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kangsoo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerd Bruder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gregory F. Welch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Austin Erickson .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA

    Jessie Y. C. Chen

  2. U.S. Army Combat Capabilities Development Command Soldier Center, Orlando, FL, USA

    Gino Fragomeni

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Erickson, A., Kim, K., Bruder, G., Welch, G.F. (2021). Beyond Visible Light: User and Societal Impacts of Egocentric Multispectral Vision. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. HCII 2021. Lecture Notes in Computer Science(), vol 12770. Springer, Cham. https://doi.org/10.1007/978-3-030-77599-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77599-5_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77598-8

  • Online ISBN: 978-3-030-77599-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation