Abstract
Today’s camera systems used for machine vision and scientific applications have intra-scene dynamic ranges up to 16 bit and therefore A/D converters with up to 16 bit resolution per pixel. Unfortunately, the linear amplification of electrons also forces a linear or even quadratic increase of the image noise variance with the signal. Based on a method published in 2016 (B. Jähne, M. Schwarzbauer, tm-Technisches Messen 83.1), this paper describes a more general nonlinear transformation which equalizes the combined effect of temporal noise and photo-response non-uniformity (PRNU) and/or temporal noise in the illumination system of an image sensor. With this generalisation it is possible to use the equalisation also for microscopic applications for which an example is discussed.
Zusammenfassung
Heutige Kamerasysteme, die für maschinelles Sehen oder wissenschaftliche Anwendungen eingesetzt werden, haben aufgrund ihrer hohen Intra-Szenen Dynamik eine digitale Auflösung von bis zu 16 Bit pro Pixel. Leider verursacht die lineare Verstärkung der Elektronen auch einen linearen oder quadratischen Anstieg der Varianz des Rauschens mit dem Signal. Basierend auf einer in 2016 veröffentlichten Methode (B. Jähne, M. Schwarzbauer, tm-Technisches Messen 83.1) beschreibt dieser Artikel eine nichtlineare Transformation, die den kombinierten Effekt von zeitlichem Rauschen, PRNU und/oder zeitlichem Rauschen im Beleuchtungssystem eines Bildsensors equilibriert. Durch diese Generalisierung ist es möglich, die Equilibrierung auch in Mikroskopie-Anwendungen einzusetzen, was mithilfe eines Beispiels gezeigt wird.
Correction note
Correction added after online publication May 23, 2019: Mistakenly this article was previously published online ahead of print omitting 2 authors’ names: Fred S. Wouters and Stephan Hock.
Funding source: Bundesministerium für Bildung und Forschung
Award Identifier / Grant number: 13N14371
Award Identifier / Grant number: 13N14373
Funding statement: The authors gratefully acknowledge financial support for this research by the research grant of the German government: 13N14371, 13N14373. As well as by the Heidelberg Collaboratory for Image Processing (HCI) within the Institutional Strategy ZUK49 “Heidelberg: Realizing the Potential of a Comprehensive University”, Measure 6.4 including matching funds from the industry partners of the HCI, especially PCO AG.
About the authors
Daniel M. Kirchhöfer received his Bachelor degree in Mechatronics from Technical University of Applied Sciences, Regensburg, Germany, in 2014 and his Master degree in Applied Research in Engineering Sciences from Technical University of Applied Sciences, Ingolstadt, Germany, in 2015. He joined PCO AG, Kelheim, Germany, in 2015 as an R&D software engineer. His main focus at PCO is software for quality control and 3rd party software, including SDK and API. In 2016 he joined the EMVA1288 working group hosted by the European Machine Vision Association. In 2017 he has started working towards the Ph.D. degree. His research interests include image compression algorithm for very high dynamic image data, image processing and advanced image visualization methods.
Gerhard A. Holst received his Diploma degree in communications engineering from the RWTH Technical University of Aachen in 1991 and his Doctoral degree in communications engineering from the University of Dortmund in 1994. From 1994 to 2001 he worked as a scientist at the Max-Planck-Institute for Marine Microbiology within the microsensor research group, where he investigated optical chemical microsensor and camera based systems. In 1998 he co-founded the company Presens (www.presens.de) in Regensburg, which he left in 2003. Since 2001 he is with PCO AG, being the head of the science & research department. His research interests are scientific camera systems, image processing, fluorescence lifetime imaging, and their applications.
Fred S. Wouters received his Bachelor and Doctoral degree in Medical Biology from Utrecht University, the Netherlands in 1992 and 1997, respectively. He received his Habilitation degree in Physiology from the medical faculty of Göttingen University in 2006. From 2007, he holds a professorship in Molecular Microscopy at the University Medical Center, Göttingen where he leads the research group Molecular and Cellular Systems. His research interests include the imaging of cellular signaling reactions and pathological tissues using fluorescence lifetime microscopy, 3D fluorescence microscopies including mesoscopic light sheet microscopy and image processing.
Stephan Hock received his Bachelor degree in Transport System Engineering in 2011 and his Master degree in Aeronautics and Astronautics in 2013, both at the Technical University of Berlin, Germany. In 2013, he started working for Ariane Group GmbH (then Astrium GmbH) in Lampoldshausen, Germany. Until 2916, he performed experiments in the German Aerospace Center in Göttingen, Germany, for his PhD thesis in Physics at the Justus-Liebig-University, Gießen, Germany. From 2016 to 2017 he continued to work for Ariane Group as a test engineer for electric space propulsion systems. Since 2018 he works as a researcher at University Medicine Göttingen, Germany, department of neuropathology in the research groups of Prof. Fred Wouters and Dr. Gertrude Bunt. His work is focussed on 3D microscopy imaging systems for pathology applications, namely hardware requirements and image data processing for contrast enhancement and quantitative methods.
Bernd Jähne received his Diploma, Doctoral degree and Habilitation degree in Physics from Heidelberg University in 1977, 1980, and 1985, respectively, and a Habilitation degree in Applied Computer Science from the University of Hamburg-Harburg in 1992. From 1988 to 2003 he hold a research professorship at the Scripps Institution of Oceanography, University of California in San Diego. Since 1994 he is professor at the Interdisciplinary Center for Scientific Computing (IWR) and Institute for Environmental Physics of Heidelberg University. From 2008 to 2017 he headed the Heidelberg Collaboratory for Image Processing (HCI). Now he is senior professor at the HCI. His research interests include image processing and small-scale air-sea interaction. He also chairs the EMVA 1288 standardization committee of the European Machine Vision Association (EMVA) for camera characterization.
Author contributions: DK, GH and BJ contributed in the mathematical and processing related equilibration method as well as the inverse equilibration, incl. the comparison and noise graphics. FW and SH contributed all medical application related processes and data. DK performed the main writing, all other authors contributed.
Acknowledgment
We thank the Technology Platform Clinical Optical Microscopy (CLINOMIC) of the University Medical Center Göttingen, headed by Dr. Gertrude Bunt for support with light sheet imaging.
References
1. Arnaud Darmont et al. “Implementing and using the EMVA1288 standard”. In: Sensors, Cameras, and Systems for Industrial and Scientific Applications XIII, Vol. 8298. International Society for Optics and Photonics. 2012, 82980H.10.1117/12.905874Search in Google Scholar
2. Christoph Drobek, Robert Mau, and Hermann Seitz. “Tomographic particle image velocimetry of a water-jet for low volume harvesting of fat tissue for regenerative medicine”. In: Current Directions in Biomedical Engineering 1.1 (2015), pp. 345–348.10.1515/cdbme-2015-0085Search in Google Scholar
3. EMVA 1288 Working Group. EMVA Standard 1288 – Standard for Characterization of Image Sensors and Cameras, Release 3.1. Open standard. European Machine Vision Association, 2016. DOI: 10.5281/zenodo.235942.Search in Google Scholar
4. Gerald C. Holst, Terrence S. Lomheim. CMOS/CCD sensors and camera systems. 2007.Search in Google Scholar
5. Jaroslav Icha et al. “Phototoxicity in live fluorescence microscopy, and how to avoid it”. In: Bioessays 39.8 (2017), p. 1700003.10.1002/bies.201700003Search in Google Scholar PubMed
6. Bernd Jähne. Digitale Bildverarbeitung. 7th ed. Springer-Verlag, 2013.10.1007/978-3-642-04952-1Search in Google Scholar
7. Bernd Jähne and Martin Schwarzbauer. “Noise equalisation and quasi loss-less image data compression–or how many bits needs an image sensor?” In: tm-Technisches Messen 83.1 (2016), pp. 16–24.10.1515/teme-2015-0093Search in Google Scholar
8. James Janesick, Kenneth Klaasen and Tom Elliott. “CCD charge collection efficiency and the photon transfer technique”. In: Solid-State Imaging Arrays. Vol. 570, International Society for Optics and Photonics. 1985, pp. 7–20.10.1117/12.950297Search in Google Scholar
9. Daniel Kirchhöfer and Bernd Jähne. “Extended noise equalisation for image compression considering linear and quadratically increasing noise sources”. In: Forum Bildverarbeitung 2018, KIT Scientific Publishing. 2018, pp. 71–80.Search in Google Scholar
10. A. Papoulis. Probability, Random Variables and Stochastic Processes, 3rd ed. McGraw-Hill, 1991.Search in Google Scholar
11. PCO AG. pco.edge 4.2 LT Datasheet. Ed. by PCO AG. 2018. url: https://www.pco.de/fileadmin/user_upload/pco-product_sheets/pco.edge_42_lt_data_sheet.pdf (visited on 09/20/2018).Search in Google Scholar
12. Eli Peli, “Contrast in complex images”. In: JOSA A 7.10 (1990), pp. 2032–2040.10.1364/JOSAA.7.002032Search in Google Scholar
13. Markus B. Sikkel et al. “High speed sCMOS-based oblique plane microscopy applied to the study of calcium dynamics in cardiac myocytes”. In: Journal of biophotonics 9.3 (2016), pp. 311–323.10.1002/jbio.201500193Search in Google Scholar PubMed PubMed Central
14. Michael J. Taormina et al. “Investigating bacterial-animal symbioses with light sheet microscopy”. The Biological Bulletin 223.1 (2012), pp. 7–20.10.1086/BBLv223n1p7Search in Google Scholar PubMed PubMed Central
15. B. Widrow and I. Kollar. Roundoff Error in Digital Computation, Signal Processing, Control, and Communications. Cambridge University Press. 2008, 61–108.Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
