Biobeam—Multiplexed wave-optical simulations of light-sheet microscopy

Martin Weigert; Kaushikaram Subramanian; Sebastian T. Bundschuh; Eugene W. Myers; Moritz Kreysing

doi:10.1371/journal.pcbi.1006079

Abstract

Sample-induced image-degradation remains an intricate wave-optical problem in light-sheet microscopy. Here we present biobeam, an open-source software package that enables simulation of operational light-sheet microscopes by combining data from 10⁵–10⁶ multiplexed and GPU-accelerated point-spread-function calculations. The wave-optical nature of these simulations leads to the faithful reproduction of spatially varying aberrations, diffraction artifacts, geometric image distortions, adaptive optics, and emergent wave-optical phenomena, and renders image-formation in light-sheet microscopy computationally tractable.

Author summary

Modern microscopes permit to acquire high quality images of large fields of view, which is the result of a decade-long development of computer aided optical design. However, this high image quality can only be obtained at the very surface of biological specimens: when trying to penetrate deeper into biological tissues, light scattering by cells rapidly leads to severe image blur and computers have so far been unable to model the process by which light forms images in such turbid optical environments. We developed a software that allows one to simulate how microscopes record images deep inside scattering biological samples. Our software reproduces a wide range of optical effects that underlie image blur in tissues. Hence strategies to improve image quality within three-dimensional samples can now be systematically tested by computers. Specifically, our software reproduces intricate wave-optical effects that have recently been proposed as strategies to gain perfect images even in the most turbid environments.This provides the chance for a new generation of microscopes, in which computer models guide the imaging process to enable highest possible resolution even deep inside biological specimens.

Citation: Weigert M, Subramanian K, Bundschuh ST, Myers EW, Kreysing M (2018) Biobeam—Multiplexed wave-optical simulations of light-sheet microscopy. PLoS Comput Biol 14(4): e1006079. https://doi.org/10.1371/journal.pcbi.1006079

Editor: Saad Jbabdi, Oxford University, UNITED KINGDOM

Received: July 5, 2017; Accepted: March 6, 2018; Published: April 13, 2018

Copyright: © 2018 Weigert et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Our software is available from: https://maweigert.github.io/biobeam.

Funding: Funding was provided by the Max Planck Society and the Bundesministerium für Bildung und Forschung (BMBF, grant number 031L0044). The authors are responsible for the content of this publication. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

This is a PLOS Computational Biology Software paper.

Introduction

Light-sheet fluorescence microscopy is a popular tool for the volumetric imaging of developing organisms [1–5]. As light-sheet microscopes continue to be developed for progressively bigger biological samples [6, 7], there is an increasing need for computers to process gigantic sets of imaging data and extract biologically relevant information [8]. With sample size, however, also light-scattering induced imaging artifacts become increasingly prevalent. During data acquisition as well as post-processing, these imaging artifacts are mostly dealt with on a purely phenomenological basis. A faithful forward model of the wave-optical imaging process, however, would i) enable rigorous benchmarks of deconvolution and segmentation strategies against ground-truth data [9], ii) serve as training platforms for machine learning approaches for image restoration and information extraction and iii) leverage the efficient use of adaptive optics [10, 11] to prevent sample-induced image degradation during the acquisition process. Predicting light-tissue interactions is particularly demanding when leaving the single scattering regime [12] or strictly diffusive transport [13]. And despite significant computational advances [14], generally applicable solutions [15] remain computationally costly, effectively prohibiting the simulation of image formation in microscopy. Even when constraining simulations to the biological relevant case of predominantly forward scattering tissue [10, 16], individual point spread function (PSF) calculations still require multiple seconds [17]. As aberrations deep inside tissues are unique for virtually each point in the sample [10, 12] a realistically large biological specimen, i.e. an embryo with a volume ∼ 100 μm³ would require 10⁵–10⁶ volumetric PSF calculations in order to faithfully mimic the wave-optical imaging process, which with current methods would take several weeks. As a consequence, attempts to simulate microscopic imaging have been limited to ray optics [18] or convolution with a constant PSF [19], approaches that do not reflect the wave-optical nature of light interaction with optically heterogeneous biological samples. Here, we report on biobeam, a software package that enables the first rigorous simulations of wave-optical image formation in light-sheet microscopes by i) a novel multiplexing scheme for PSF calculations and ii) efficient GPU parallelization.

Design and implementation

The pipeline underlying our software is based on the observation that the beam-propagation model (BPM) for fiber optics [20, 21] can also be used to mimic scattering biological cells [22]. To guarantee good accuracy of BPM beyond strictly paraxial wave propagation, we use the exact propagator together with a locally adapted expansion of refractive indices (see S1 Text, Notes 1 and 4 for details and validation against analytically tractable scattering models). To massively reduce the computational cost for ∼10⁶ wave-optical PSF calculations, we introduce a novel multiplexing scheme. In this we exploit the fact that for typical imaging scenarios, a single camera image can be constructed from sets of 100s to 1000s of mutually independent, spatially varying, non-overlapping PSFs. Such sets can be calculated within single, highly multiplexed simulations (see S3 Fig and S1 Text, Note 8), which is similar to the operational principle of highly multiplexed confocal measurements of spinning disc microscopes [23, 24]. Our software is further accelerated ∼20 fold by the efficient use of GPU implementations for all low-level calculations. In this way, the propagation of an arbitrary light field such as the multiplexed set of ∼1000 PSFs through a typical volume of 1024³ voxels takes less than 500ms on a single graphics card (cf. S1 Text, Note 5, S1 Table and S4 Video), corresponding to a 20.000 fold acceleration compared to the sequential calculation of PSFs. Implementation was done as an open source Python software package. All computationally heavy parts are lifted to the GPU via OpenCL [25], thereby keeping all the advantages of Python as a dynamically typed high level language that is vastly used in the scientific community without compromising on performance. Apart from its technical focus on speed, biobeam is specifically designed to make wave-optical experiments in-silico as easy as possible, e.g. by providing a simple API that makes it easy to apply different input fields as well as PSF/aberration calculations via diffraction-limited point source propagation through tissue (see S1 Text, Note 3 for example listings).

Results

Light-sheet microscopy

We first demonstrate the power of our software package by presenting the first wave-optical simulation of the volumetric image-formation process in a light-sheet microscope [5, 9, 18]. To start, a cylindrical sheet of light is propagated at a specific axial position through a fluorescently labelled embryo model with refractive indices of cytoplasm, nuclei and organelles as previously reported [12] (Fig 1a and S1 & S7 Videos). From this, one obtains the fluorescence excitation at every point in the volume for every axial position of the incident light field. Next, for each focal plane a full set of detection PSFs is obtained by propagating light from multiplexed, diffraction-limited point-sources orthogonally through an idealized, refocusing lens towards the camera (Fig 1c, upper left, S2 Video). This way, we obtain a quasi-continuum of spatially-dependent, volumetric PSFs with position-dependent aberrations that stem from distortions and scattering in both the illumination and the detection paths (Fig 1c left, see also S4 Fig for calculated aberration maps). Convolving the exhaustively sampled sets of spatially-varying PSFs with the fluorescent object finally yields the wave-optical image as seen by the camera (Fig 1c, right). These simulations are particularly demanding due to the large grid size (i.e. 1024 × 2048 × 1024 voxels for Fig 1) and sampling density requirements (typically 0.5–5μm), here resulting in a total of 10⁶ PSFs that were calculated in well under 10 minutes on a single graphics card (see also S1 and S2 Videos). In contrast, the non-multiplexed calculations would have required more than two months on a single CPU. As a result of this computational pipeline, biobeam generates faithfully calculated 3D microscopy data sets that account for both refraction and diffraction based imaging artifacts. These include image blur and contrast loss, spatially varying PSFs, speckle artifacts, image granularity, as well as sample-induced geometric distortions such as lensing, image displacements and split-screen type double images (cf. Fig 2a and 2b, S3 Video and S1 Fig for benchmarks against analytically tractable models).

Download:

Fig 1. Rigorous wave-optical simulation of image formation process in light-sheet microscopy.

(a) Synthetic tissue phantom of a multicellular organism (100 × 200 × 100μm) comprising a complex refractive index distribution (left, n = 1.33–1.38) and a fluorophore distribution of interest (right). (b) Wave optical simulation of the illuminating light sheet and resulting excitation distribution within the sample at a given z position. (c) Partially coherent simulation of the detection path by multiplexed calculation of all independent point spread functions (left) and the resulting simulated camera image combining illumination and fluorescence path of light through the scattering sample. (d) Alternative light-sheet modalities (see also S1 and S5 Videos and main text for details).

https://doi.org/10.1371/journal.pcbi.1006079.g001

Download:

Fig 2. Optical capabilities of the biobeam image-formation pipeline.

(a) A test chart at the mid-section of an optically heterogeneous embryo-model (n = 1.35–1.39 diameter 140μm) is illuminated by cylindrical light sheet (NA = 0.15), and imaged from an orthogonal position (NA = 0.6). (b) Details of these wave-optically calculated images reveal i) spatially varying image blur, contrast loss and absorption induced by the heterogeneity of the sample, ii) diffraction artifacts from the light-sheet-typical coherent illumination, and iii) geometric image distortions such as lensing, split-screen type image distortions, and object displacements lensing. (c) Biobeam is further capable of adaptive optics simulations by which reversal of guide star emitted light fields yields perfect foci in scattering tissues. (d) Adaptive optics simulations faithfully reproduces the shift-shift memory effect, an emergent wave-optical phenomenon, here at 4 mean-free-paths deep inside the tissue.

https://doi.org/10.1371/journal.pcbi.1006079.g002

We additionally validated our approach by comparing simulation results for several experimental situations, such as the diffraction pattern behind an edge knife (S5 Fig), the scattering of a light-sheet at agarose embedded micro-spheres (S6 Fig) and the image distortions obtained when projecting test images through a refracting sphere (S7 and S8 Figs). Beyond the theoretical validation of the core propagation model, these successful comparisons of simulations against experiments further demonstrate that our high-level implementation works correctly in the biological relevant case of low refractive index contrasts.

Being a particularly flexible software package, biobeam provides further pre-implemented illumination modalities, including Bessel lattices (Fig 1d, right, see S2 Fig, S5 Video), and allows for practical extraction of sample-induced aberrations as spatially resolved Zernike maps (see S4 Fig and S1 Text, Note 9).

Adaptive optics simulations

Next, we demonstrate biobeam’s capability to accurately simulate wave-optical effects relevant to adaptive optics (AO) imaging [10]. It is well-established and exploited in imaging that perfect imaging foci can be created behind strongly scattering screens [26] and within biological samples by appropriately shaping the wavefront before entry into the scattering medium. We recapitulated this finding by explicit simulations of light propagating through a spatially extended synthetic tissue sample represented by a Perlin-noise refractive index distribution (n = 1.36 ± 0.03). As expected, we find also in simulations that conjugation of the wavefront at the surface of the sample allows one to generate diffraction-limited foci inside these scattering tissues (Fig 2c, S6 Video and S9 Fig). Furthermore, we show that biobeam is capable of reproducing the shift-shift memory effect, an emergent wave-optical phenomenon responsible for the significant robustness of adaptive imaging against lateral focus displacements. According to this wave-optical phenomenon, the radius of the iso-planatic patch (distance over which an AO correction pattern works) is determined by the statistics of a tissue generated speckle field inside this medium. Based on a total of 22500 PSF calculations, we show that our simulations faithfully capture this emergent wave-optical effect. In agreement with recent analytical arguments and their empirical confirmation [10], we show by our computational microscopy experiments that the average persistence of the laterally shifted focus is precisely limited by the autocorrelation length of a speckle pattern that would result from an incident plane wave. This phenomenon is accurately reproduced even four mean-free-path lengths inside the tissue (Fig 2d). While currently using a variant of the beam propagation method, our PSF multiplexing scheme is in principle also compatible with other low-level field stepping algorithms, including those that iteratively account for multiple back-reflections [14, 15], when even higher precision or penetration depths are required.

Discussion

We summarize that biobeam enables faithful whole-tissue wave-optical simulations of light-sheet microscopes due to i) a novel multiplexing scheme of PSF calculations and ii) efficient GPU parallelization. Biobeam renders the biological imaging process computationally tractable, thus providing the link between wave-optically recorded image material, and the ground truth object. Given the modular nature of our software package, these simulations are easy to implement and can be flexibly adapted to custom imaging scenarios and microscopes. Beyond the reproduction and identification of commonly occurring imaging artifacts (see Supplement for comparison with experimental data), we demonstrated that biobeam is compatible with imaging scenarios in which sample-induced PSFs degradation is overcome by the use of adaptive optics. Furthermore, we showcase the optical capabilities and accuracy of our software by explicitly demonstrating that emergent wave-optical phenomena such as the shift-shift memory effect are quantitatively reproduced deep inside tissues. While we chose a variant of BPM as a low-level field stepping routine, the here presented strategy of multiplexing PSF calculation is more general in nature, and may also be used in combination with other light propagation algorithms, e.g. in scenarios where higher accuracy at large angles and/or more isotropic scattering is of relevance [14].

We conclude that biobeam is a flexible and particularly powerful platform to systematically study wave-optical image-formation by microscopes in scattering biological tissues.

Availability and future directions

Prospectively, we see biobeam helping to improve microscope design, enhancing deconvolution and segmentation strategies by providing realistic imaging data-sets along with ground truth data, and paving the way for a new generation of smart, adaptive microscopes that learn to treat the sample as a part of the optical path. Of great help in this will be the rapidly increasing knowledge of stereotypical refractive index distributions in embryos, tissues, cells, subcellular compartments and their constituents as derived from tomographic phase microscopy [12, 27–29] and complemented by electron microscopy morphological data [30]. Beyond the simulation of light-sheet microscopes, our software can also be used to simulate other imaging modalities such as wide-field, laser-scanning confocal, and light-field [31] microscopes, as well as for novel micro-lens concepts [32] and the emerging field of soft photonics [33]. Especially, we emphasize the potential to improve the understanding of physiological image-formation inside the eye [34], taking into account the optics of the lens [35] as well as the retina [36–39]. Biobeam is available as open-source (BSD-3 license) python package at https://maweigert.github.io/biobeam. Datasets can be found at https://publications.mpi-cbg.de/6874-data/.

Supporting information

S1 Text. 25 pages description of the numerical method, validation, benchmarks, details of the numerical experiments, and example listings.

https://doi.org/10.1371/journal.pcbi.1006079.s001

(PDF)

S1 Video. Wave-optical simulation of the image-formation process in light-sheet microscopy.

The tissue model represents a multicellular (760 nuclei) organism of size (100μm, 200μm, 100μm) in an aqueous medium with n = 1.33. The refractive index distribution is in the range n ∈ (1.35, 1.42) comprising reference values for cell nuclei, eggshell and the cytoplasm [12]. Weak absorption is homogenously present, but could also be localized (e.g. a spherical absorbing compartment in the center). The simulations of both the illumination and detection processes were carried out on a computational grid of (1024, 2048, 1024) voxels with a spacing of 100nm along each dimension. The illumination field is a cylindrical light sheet with NA_illum = 0.1 focused laterally at the center and the detection system was assumed to have NA_detect = 0.6. For generating the final stack both illumination and detection fields were simulated at 200 different axial positions. The deterioration of both resolution and intensity at regions where photons along either the illumination or detection path had to travel through large inhomogeneities can clearly be seen.

https://doi.org/10.1371/journal.pcbi.1006079.s002

(MP4)

S2 Video. Illustration of a single PSF calculation inside the tissue via the propagation of analytically defined diffraction-limited input fields.

Due to the linearity of wave-optics, these PSF calculations can be highly multiplexed, as firstly exploited by biobeam.

https://doi.org/10.1371/journal.pcbi.1006079.s003

(MP4)

S3 Video. A biobeam generated video illustrating rigorous wave-optical mimicry of a wide-field microscope.

The imaging of a 100 μm² test chart is simulated while a refractive sphere is continuously introduced into the microscope’s optical path. biobeam generated the underlying wave-optical simulations in 30 seconds.

https://doi.org/10.1371/journal.pcbi.1006079.s004

(MP4)

S4 Video. Screencast of an interactive command line session demonstrating biobeam’s capabilities and speed.

All calculations happen in real time.

https://doi.org/10.1371/journal.pcbi.1006079.s005

(MP4)

S5 Video. Video showing the predefined illumination modes and simulated light sheets being scanned through a biologically plausible tissue model.

Both coherent (cylindrical lens SPIM) illumination and partially-incoherent illumination modes (time scanned Gaussian/Bessel beams) are simulated.

https://doi.org/10.1371/journal.pcbi.1006079.s006

(MP4)

S6 Video. Showing the simulation of a aberration pre-compensated wavefront focusing deep into tissue and the shift-shift memory effect.

https://doi.org/10.1371/journal.pcbi.1006079.s007

(MP4)

S7 Video. Example of a plane-by-plane illumination of a tissue model mimicking an embryo.

https://doi.org/10.1371/journal.pcbi.1006079.s008

(MP4)

S1 Fig. Validation of biobeam with analytical solutions.

a) Plane wave scattered by three solid spheres (λ = 500nm, r = 2-2.5μm, refractive index contrast m = 1.05), b) Comparison of analytical solution (Mie calculus) versus biobeam simulation. c) Error percentage of near field distribution as a function of single sphere radius r (Δn = 0.05) and refractive index contrast Δn (r = 2.5μm). d) Left: Phase function of analytically tractable coated spheres as cell models (m = 1.02/1.04, r = 5μm/4μm) shows high accuracy up to approximately 0.5 radians. Right: size dependent scattering efficiency of the same sphere architecture and its inverse.

https://doi.org/10.1371/journal.pcbi.1006079.s009

(PDF)

S2 Fig. Propagation of different predefined input fields through a tissue model of size (100μm, 100μm, 100μm) and grid dimension (1024³).

The respective pupil function is shown in the upper row.

https://doi.org/10.1371/journal.pcbi.1006079.s010

(PDF)

S3 Fig. Detection aberration and PSF calculation.

Propagating a diffraction limited input field through parts of the sample and refocusing by an idealized optical system gives the focus field as seen by the detector. If the refocus spots are separated for different starting points, the propagation of a complete grid can be carried out in a highly multiplexed manner, accelerating the process for typical microscopy simulations by a factor 100–1000.

https://doi.org/10.1371/journal.pcbi.1006079.s011

(PDF)

S4 Fig. Calculating the aberrations of the detection PSF for a given z plane within a synthetic tissue model.

The model’s physical size is (200μm, 100μm, 100μm) and the dimension of the computational grid is (1024, 512, 512). The detection wavelength is λ = 522nm, the numerical aperture is NA = 0.5 and the aqueous immersion medium has refractive index n₀ = 1.33. The refractive index distribution of the tissue model mimics an eggshell, cell nuclei and granular random fluctuations within the biological plausible range of n ∈ (1.35, 1.43).

https://doi.org/10.1371/journal.pcbi.1006079.s012

(PDF)

S5 Fig. Diffraction around a knife edge.

a) Experimental setup: Light is focused with an incoherent light source (M470L3 Thorlabs, λ = 470nm) such that an almost plane wave (NA = 0.001) illuminated the knife edge. The diffracting light was imaged below at different depths from the edge. b) The simulation was done on a computational cell of size (1024 × 256 × 1830) with voxel size Δx = 0.29μm. We simulated the diffraction in the case of a single plane wave (coherent, top) and the incoherent superposition of 100 random incident plane waves of different small incident angle (corresponding to NA = 0.001, bottom). c) The experimental acquired intensity. Scalebar is 12μm in both axial and lateral direction (depicted with axial/lateral aspect ratio of 8, due to space constraints). d) Intensity plot at a given axial position (dashed line) for simulation, experiment and the intensity calculated via Fresnel-integral (Theory).

https://doi.org/10.1371/journal.pcbi.1006079.s013

(PDF)

S6 Fig. Experimental validation on a commercial light-sheet microscope.

a) Poly-methylmethacrylate (PMMA) micro-particles with a diameter of 20μm and refractive index of n = 1.495 were embedded in an OptiPrep/agarose block (n ≈ 1.43) labelled with Alexa Fluor 488. A stationary illuminating light-sheet with a waist of 1.7μm and a lateral extension of ≈100μm was generated with a LZ1 (Zeiss) light-sheet microscope, incident on the agarose embedded spheres. Stacks were acquired at a step size of 0.414μm. b) Simulation results of the intensity distribution behind the sphere at a plane incident to the sphere center. c) Experimental intensity image. Scalebar is 20μm in both cases. Dashed lines indicate regions with specific diffraction patterns that the simulation correctly reconstitutes.

https://doi.org/10.1371/journal.pcbi.1006079.s014

(PDF)

S7 Fig. Custom micro-projection setup built in our lab, controlled with custom LabView programs.

The setup allows for patterns to be micro projected onto a sample with predefined illumination-source, size, magnification and NA of influx optics. The efflux optics allows for the collection of the light and recording on the camera.

https://doi.org/10.1371/journal.pcbi.1006079.s015

(PDF)

S8 Fig. Experimental micro-projection of a test-chart through a glass sphere and comparison with simulation.

Experimental setup as in S7 Fig. The negative USAF (R1DS1N, Thorlabs) test-chart was illuminated incoherently (M470L3 Thorlabs) and projected behind a glass sphere (Borosilicate material, n = 1.48, 110μm diameter, Cospheric LLC, USA). The images were captured using an Andor Zyla 5.5 sCMOS camera, while focusing through the sphere (see S7 Fig). Depicted are images from the experiment (Real) and the simulation. The difference images are calculated wrt. to the undistorted test-chart image, showing that the real sphere-induced image distortions are qualitatively reproduced by the simulation.

https://doi.org/10.1371/journal.pcbi.1006079.s016

(PDF)

S9 Fig. Simulation of shift-shift memory effect.

a) Guide-star assisted diffraction-limited focusing in scattering tissue model. b) Lateral translation of this aberration compensated beam leads to gradual degradation. c) Quantification of focus degradation via correlation C(Δx) vs. distance (dotted line) at different penetration depth (in mean free path). Agreement with the correlation function of a scattered plane wave (solid line) is most pronounced at higher penetration depth. Deviation at low penetration depth (1 mfp) agree with experimental observations [10].

https://doi.org/10.1371/journal.pcbi.1006079.s017

(PDF)

S10 Fig. Aberration correction.

a) A rectilinear ground truth stripe pattern occurs distorted when imaged through a sphere. b) Fitting a radial distortion map to simulation results of this scenario, allows one to fix the aberrations at the region of interest (green) as seen through the spherical cell phantom.

https://doi.org/10.1371/journal.pcbi.1006079.s018

(PDF)

S1 Table. Runtimes of different methods for the simulation of a plane wave propagating through a given refractive index distribution.

https://doi.org/10.1371/journal.pcbi.1006079.s019

(PDF)

Acknowledgments

We thank Benjamin Judkewitz (Charité Berlin), Pavel Tomancak, Loic Royer (all MPI-CBG) for helpful discussions and insightful comments.

References

1. Power RM, Huisken J. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nature Methods. 2017;14(4):360–373. pmid:28362435
- View Article
- PubMed/NCBI
- Google Scholar
2. Stelzer EH. Light-sheet fluorescence microscopy for quantitative biology. Nature methods. 2015;12(1):23–28. pmid:25549266
- View Article
- PubMed/NCBI
- Google Scholar
3. Tomer R, Khairy K, Amat F, Keller PJ. Quantitative high-speed imaging of entire developing embryos with simultaneous multiview light-sheet microscopy. Nature methods. 2012;9(7):755–763. pmid:22660741
- View Article
- PubMed/NCBI
- Google Scholar
4. Huisken J, Stainier DY. Selective plane illumination microscopy techniques in developmental biology. Development. 2009;136(12):1963–1975. pmid:19465594
- View Article
- PubMed/NCBI
- Google Scholar
5. Huisken J, Swoger J, Del Bene F, Wittbrodt J, Stelzer EH. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science. 2004;305(5686):1007–1009. pmid:15310904
- View Article
- PubMed/NCBI
- Google Scholar
6. Strnad P, Gunther S, Reichmann J, Krzic U, Balazs B, de Medeiros G, et al. Inverted light-sheet microscope for imaging mouse pre-implantation development. Nature methods. 2016;13(2):139–142. pmid:26657559
- View Article
- PubMed/NCBI
- Google Scholar
7. Economo MN, Clack NG, Lavis LD, Gerfen CR, Svoboda K, Myers EW, et al. A platform for brain-wide imaging and reconstruction of individual neurons. Elife. 2016;5:e10566. pmid:26796534
- View Article
- PubMed/NCBI
- Google Scholar
8. Meijering E, Carpenter AE, Peng H, Hamprecht FA, Olivo-Marin JC. Imagining the future of bioimage analysis. Nature Biotechnology. 2016;34(12):1250–1255. pmid:27926723
- View Article
- PubMed/NCBI
- Google Scholar
9. Preibisch S, Amat F, Stamataki E, Sarov M, Singer RH, Myers E, et al. Efficient Bayesian-based multiview deconvolution. Nature Methods. 2014;11(6):645–648. pmid:24747812
- View Article
- PubMed/NCBI
- Google Scholar
10. Judkewitz B, Horstmeyer R, Vellekoop IM, Papadopoulos IN, Yang C. Translation correlations in anisotropically scattering media. Nature physics. 2015;11(8):684–689.
- View Article
- Google Scholar
11. Wang K, Milkie DE, Saxena A, Engerer P, Misgeld T, Bronner ME, et al. Rapid adaptive optical recovery of optimal resolution over large volumes. Nature Methods. 2014;11(6):625–628. pmid:24727653
- View Article
- PubMed/NCBI
- Google Scholar
12. Choi W, Fang-Yen C, Badizadegan K, Oh S, Lue N, Dasari RR, et al. Tomographic phase microscopy. Nature Methods. 2007;4:717–719. pmid:17694065
- View Article
- PubMed/NCBI
- Google Scholar
13. Zhu C, Liu Q. Review of Monte Carlo modeling of light transport in tissues. Journal of Biomedical Optics. 2013;18(5):050902.
- View Article
- Google Scholar
14. Osnabrugge G, Leedumrongwatthanakun S, Vellekoop IM. A convergent Born series for solving the inhomogeneous Helmholtz equation in arbitrarily large media. Journal of computational physics. 2016;322:113–124.
- View Article
- Google Scholar
15. Oskooi AF, Roundy D, Ibanescu M, Bermel P, Joannopoulos JD, Johnson SG. Meep: A flexible free-software package for electromagnetic simulations by the FDTD method. Computer Physics Communications. 2010;181(3):687.
- View Article
- Google Scholar
16. Jacques SL. Optical properties of biological tissues: a review. Physics in Medicine and Biology. 2013;58(11):R37. pmid:23666068
- View Article
- PubMed/NCBI
- Google Scholar
17. Glaser AK, Chen Y, Liu JT. Fractal propagation method enables realistic optical microscopy simulations in biological tissues. Optica. 2016;3(8):861–869. pmid:28983499
- View Article
- PubMed/NCBI
- Google Scholar
18. Uddin MS, Lee HK, Preibisch S, Tomancak P. Restoration of uneven illumination in light sheet microscopy images. Microscopy and Microanalysis. 2011;17(04):607–613. pmid:21682937
- View Article
- PubMed/NCBI
- Google Scholar
19. Venkataramani V, Herrmannsdörfer F, Heilemann M, Kuner T. SuReSim: simulating localization microscopy experiments from ground truth models. Nature Methods. 2016;13(4):319–321. pmid:26928761
- View Article
- PubMed/NCBI
- Google Scholar
20. Kawano K, Kitoh T. Introduction to optical waveguide analysis. New York: John Wiley & Sons; 2001.
21. Van Roey J, Van der Donk J, Lagasse P. Beam-propagation method: analysis and assessment. J Opt Soc Am. 1981;71(7):803–810.
- View Article
- Google Scholar
22. Fahrbach FO, Simon P, Rohrbach A. Microscopy with self-reconstructing beams. Nature Photonics. 2010;4(11):780–785.
- View Article
- Google Scholar
23. Gräf R, Rietdorf J, Zimmermann T. Live cell spinning disk microscopy. In: Microscopy techniques. Springer; 2005. p. 57–75.
24. Petráň M, Hadravskỳ M, Egger MD, Galambos R. Tandem-scanning reflected-light microscope. JOSA. 1968;58(5):661–664.
- View Article
- Google Scholar
25. Klöckner A, Pinto N, Lee Y, Catanzaro B, Ivanov P, Fasih A. PyCUDA and PyOpenCL: A Scripting-Based Approach to GPU Run-Time Code Generation. Parallel Computing. 2012;38(3):157–174.
- View Article
- Google Scholar
26. Vellekoop IM, Mosk A. Focusing coherent light through opaque strongly scattering media. Optics Letters. 2007;32(16):2309–2311. pmid:17700768
- View Article
- PubMed/NCBI
- Google Scholar
27. Schürmann M, Cojoc G, Girardo S, Ulbricht E, Guck J, Müller P. Three-dimensional correlative single-cell imaging utilizing fluorescence and refractive index tomography. Journal of biophotonics. 2017;. pmid:28800386
- View Article
- PubMed/NCBI
- Google Scholar
28. Kim K, Park WS, Na S, Kim S, Kim T, Do Heo W, et al. Correlative three-dimensional fluorescence and refractive index tomography: bridging the gap between molecular specificity and quantitative bioimaging. Biomedical optics express. 2017;8(12):5688–5697. pmid:29296497
- View Article
- PubMed/NCBI
- Google Scholar
29. Lošt’ák M, Chmelík R, Slabá M, Slabỳ T. Coherence-controlled holographic microscopy in diffuse media. Optics express. 2014;22(4):4180–4195. pmid:24663742
- View Article
- PubMed/NCBI
- Google Scholar
30. Arnold J, Mahamid J, Lucic V, de Marco A, Fernandez JJ, Laugks T, et al. Site-specific cryo-focused ion beam sample preparation guided by 3D correlative microscopy. Biophysical journal. 2016;110(4):860–869. pmid:26769364
- View Article
- PubMed/NCBI
- Google Scholar
31. Prevedel R, Yoon YG, Hoffmann M, Pak N, Wetzstein G, Kato S, et al. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nature methods. 2014;11(7):727–730. pmid:24836920
- View Article
- PubMed/NCBI
- Google Scholar
32. Nagelberg S, Zarzar LD, Nicolas N, Subramanian K, Kalow JA, Sresht V, et al. Reconfigurable and responsive droplet-based compound micro-lenses. Nature Communications. 2017;8. pmid:28266505
- View Article
- PubMed/NCBI
- Google Scholar
33. Kolle M, Lee S. Progress and Opportunities in Soft Photonics and Biologically Inspired Optics. Advanced Materials. 2017;. pmid:29057519
- View Article
- PubMed/NCBI
- Google Scholar
34. Land MF, Nilsson DE. Animal eyes. Oxford University Press; 2012.
35. Greiling TM, Clark JI. The transparent lens and cornea in the mouse and zebra fish eye. In: Seminars in cell & developmental biology. vol. 19. Elsevier; 2008. p. 94–99.
36. Franze K, Grosche J, Skatchkov SN, Schinkinger S, Foja C, Schild D, et al. Müller cells are living optical fibers in the vertebrate retina. Proceedings of the National Academy of Sciences. 2007;104(20):8287–8292.
- View Article
- Google Scholar
37. Solovei I, Kreysing M, Lanctôt C, Kösem S, Peichl L, Cremer T, et al. Nuclear architecture of rod photoreceptor cells adapts to vision in mammalian evolution. Cell. 2009;137(2):356–368. pmid:19379699
- View Article
- PubMed/NCBI
- Google Scholar
38. Kreysing M, Boyde L, Guck J, Chalut KJ. Physical insight into light scattering by photoreceptor cell nuclei. Optics letters. 2010;35(15):2639–2641. pmid:20680084
- View Article
- PubMed/NCBI
- Google Scholar
39. Błaszczak Z, Kreysing M, Guck J. Direct observation of light focusing by single photoreceptor cell nuclei. Optics express. 2014;22(9):11043–11060. pmid:24921803
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Power RM, Huisken J. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nature Methods. 2017;14(4):360–373. pmid:28362435
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Stelzer EH. Light-sheet fluorescence microscopy for quantitative biology. Nature methods. 2015;12(1):23–28. pmid:25549266
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Tomer R, Khairy K, Amat F, Keller PJ. Quantitative high-speed imaging of entire developing embryos with simultaneous multiview light-sheet microscopy. Nature methods. 2012;9(7):755–763. pmid:22660741
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Huisken J, Stainier DY. Selective plane illumination microscopy techniques in developmental biology. Development. 2009;136(12):1963–1975. pmid:19465594
View Article
PubMed/NCBI
Google Scholar

[14] View Article

[15] PubMed/NCBI

[16] Google Scholar

[ref5] 5. Huisken J, Swoger J, Del Bene F, Wittbrodt J, Stelzer EH. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science. 2004;305(5686):1007–1009. pmid:15310904
View Article
PubMed/NCBI
Google Scholar

[18] View Article

[19] PubMed/NCBI

[20] Google Scholar

[ref6] 6. Strnad P, Gunther S, Reichmann J, Krzic U, Balazs B, de Medeiros G, et al. Inverted light-sheet microscope for imaging mouse pre-implantation development. Nature methods. 2016;13(2):139–142. pmid:26657559
View Article
PubMed/NCBI
Google Scholar

[22] View Article

[23] PubMed/NCBI

[24] Google Scholar

[ref7] 7. Economo MN, Clack NG, Lavis LD, Gerfen CR, Svoboda K, Myers EW, et al. A platform for brain-wide imaging and reconstruction of individual neurons. Elife. 2016;5:e10566. pmid:26796534
View Article
PubMed/NCBI
Google Scholar

[26] View Article

[27] PubMed/NCBI

[28] Google Scholar

[ref8] 8. Meijering E, Carpenter AE, Peng H, Hamprecht FA, Olivo-Marin JC. Imagining the future of bioimage analysis. Nature Biotechnology. 2016;34(12):1250–1255. pmid:27926723
View Article
PubMed/NCBI
Google Scholar

[30] View Article

[31] PubMed/NCBI

[32] Google Scholar

[ref9] 9. Preibisch S, Amat F, Stamataki E, Sarov M, Singer RH, Myers E, et al. Efficient Bayesian-based multiview deconvolution. Nature Methods. 2014;11(6):645–648. pmid:24747812
View Article
PubMed/NCBI
Google Scholar

[34] View Article

[35] PubMed/NCBI

[36] Google Scholar

[ref10] 10. Judkewitz B, Horstmeyer R, Vellekoop IM, Papadopoulos IN, Yang C. Translation correlations in anisotropically scattering media. Nature physics. 2015;11(8):684–689.
View Article
Google Scholar

[38] View Article

[39] Google Scholar

[ref11] 11. Wang K, Milkie DE, Saxena A, Engerer P, Misgeld T, Bronner ME, et al. Rapid adaptive optical recovery of optimal resolution over large volumes. Nature Methods. 2014;11(6):625–628. pmid:24727653
View Article
PubMed/NCBI
Google Scholar

[41] View Article

[42] PubMed/NCBI

[43] Google Scholar

[ref12] 12. Choi W, Fang-Yen C, Badizadegan K, Oh S, Lue N, Dasari RR, et al. Tomographic phase microscopy. Nature Methods. 2007;4:717–719. pmid:17694065
View Article
PubMed/NCBI
Google Scholar

[45] View Article

[46] PubMed/NCBI

[47] Google Scholar

[ref13] 13. Zhu C, Liu Q. Review of Monte Carlo modeling of light transport in tissues. Journal of Biomedical Optics. 2013;18(5):050902.
View Article
Google Scholar

[49] View Article

[50] Google Scholar

[ref14] 14. Osnabrugge G, Leedumrongwatthanakun S, Vellekoop IM. A convergent Born series for solving the inhomogeneous Helmholtz equation in arbitrarily large media. Journal of computational physics. 2016;322:113–124.
View Article
Google Scholar

[52] View Article

[53] Google Scholar

[ref15] 15. Oskooi AF, Roundy D, Ibanescu M, Bermel P, Joannopoulos JD, Johnson SG. Meep: A flexible free-software package for electromagnetic simulations by the FDTD method. Computer Physics Communications. 2010;181(3):687.
View Article
Google Scholar

[55] View Article

[56] Google Scholar

[ref16] 16. Jacques SL. Optical properties of biological tissues: a review. Physics in Medicine and Biology. 2013;58(11):R37. pmid:23666068
View Article
PubMed/NCBI
Google Scholar

[58] View Article

[59] PubMed/NCBI

[60] Google Scholar

[ref17] 17. Glaser AK, Chen Y, Liu JT. Fractal propagation method enables realistic optical microscopy simulations in biological tissues. Optica. 2016;3(8):861–869. pmid:28983499
View Article
PubMed/NCBI
Google Scholar

[62] View Article

[63] PubMed/NCBI

[64] Google Scholar

[ref18] 18. Uddin MS, Lee HK, Preibisch S, Tomancak P. Restoration of uneven illumination in light sheet microscopy images. Microscopy and Microanalysis. 2011;17(04):607–613. pmid:21682937
View Article
PubMed/NCBI
Google Scholar

[66] View Article

[67] PubMed/NCBI

[68] Google Scholar

[ref19] 19. Venkataramani V, Herrmannsdörfer F, Heilemann M, Kuner T. SuReSim: simulating localization microscopy experiments from ground truth models. Nature Methods. 2016;13(4):319–321. pmid:26928761
View Article
PubMed/NCBI
Google Scholar

[70] View Article

[71] PubMed/NCBI

[72] Google Scholar

[ref20] 20. Kawano K, Kitoh T. Introduction to optical waveguide analysis. New York: John Wiley & Sons; 2001.

[ref21] 21. Van Roey J, Van der Donk J, Lagasse P. Beam-propagation method: analysis and assessment. J Opt Soc Am. 1981;71(7):803–810.
View Article
Google Scholar

[75] View Article

[76] Google Scholar

[ref22] 22. Fahrbach FO, Simon P, Rohrbach A. Microscopy with self-reconstructing beams. Nature Photonics. 2010;4(11):780–785.
View Article
Google Scholar

[78] View Article

[79] Google Scholar

[ref23] 23. Gräf R, Rietdorf J, Zimmermann T. Live cell spinning disk microscopy. In: Microscopy techniques. Springer; 2005. p. 57–75.

[ref24] 24. Petráň M, Hadravskỳ M, Egger MD, Galambos R. Tandem-scanning reflected-light microscope. JOSA. 1968;58(5):661–664.
View Article
Google Scholar

[82] View Article

[83] Google Scholar

[ref25] 25. Klöckner A, Pinto N, Lee Y, Catanzaro B, Ivanov P, Fasih A. PyCUDA and PyOpenCL: A Scripting-Based Approach to GPU Run-Time Code Generation. Parallel Computing. 2012;38(3):157–174.
View Article
Google Scholar

[85] View Article

[86] Google Scholar

[ref26] 26. Vellekoop IM, Mosk A. Focusing coherent light through opaque strongly scattering media. Optics Letters. 2007;32(16):2309–2311. pmid:17700768
View Article
PubMed/NCBI
Google Scholar

[88] View Article

[89] PubMed/NCBI

[90] Google Scholar

[ref27] 27. Schürmann M, Cojoc G, Girardo S, Ulbricht E, Guck J, Müller P. Three-dimensional correlative single-cell imaging utilizing fluorescence and refractive index tomography. Journal of biophotonics. 2017;. pmid:28800386
View Article
PubMed/NCBI
Google Scholar

[92] View Article

[93] PubMed/NCBI

[94] Google Scholar

[ref28] 28. Kim K, Park WS, Na S, Kim S, Kim T, Do Heo W, et al. Correlative three-dimensional fluorescence and refractive index tomography: bridging the gap between molecular specificity and quantitative bioimaging. Biomedical optics express. 2017;8(12):5688–5697. pmid:29296497
View Article
PubMed/NCBI
Google Scholar

[96] View Article

[97] PubMed/NCBI

[98] Google Scholar

[ref29] 29. Lošt’ák M, Chmelík R, Slabá M, Slabỳ T. Coherence-controlled holographic microscopy in diffuse media. Optics express. 2014;22(4):4180–4195. pmid:24663742
View Article
PubMed/NCBI
Google Scholar

[100] View Article

[101] PubMed/NCBI

[102] Google Scholar

[ref30] 30. Arnold J, Mahamid J, Lucic V, de Marco A, Fernandez JJ, Laugks T, et al. Site-specific cryo-focused ion beam sample preparation guided by 3D correlative microscopy. Biophysical journal. 2016;110(4):860–869. pmid:26769364
View Article
PubMed/NCBI
Google Scholar

[104] View Article

[105] PubMed/NCBI

[106] Google Scholar

[ref31] 31. Prevedel R, Yoon YG, Hoffmann M, Pak N, Wetzstein G, Kato S, et al. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nature methods. 2014;11(7):727–730. pmid:24836920
View Article
PubMed/NCBI
Google Scholar

[108] View Article

[109] PubMed/NCBI

[110] Google Scholar

[ref32] 32. Nagelberg S, Zarzar LD, Nicolas N, Subramanian K, Kalow JA, Sresht V, et al. Reconfigurable and responsive droplet-based compound micro-lenses. Nature Communications. 2017;8. pmid:28266505
View Article
PubMed/NCBI
Google Scholar

[112] View Article

[113] PubMed/NCBI

[114] Google Scholar

[ref33] 33. Kolle M, Lee S. Progress and Opportunities in Soft Photonics and Biologically Inspired Optics. Advanced Materials. 2017;. pmid:29057519
View Article
PubMed/NCBI
Google Scholar

[116] View Article

[117] PubMed/NCBI

[118] Google Scholar

[ref34] 34. Land MF, Nilsson DE. Animal eyes. Oxford University Press; 2012.

[ref35] 35. Greiling TM, Clark JI. The transparent lens and cornea in the mouse and zebra fish eye. In: Seminars in cell & developmental biology. vol. 19. Elsevier; 2008. p. 94–99.

[ref36] 36. Franze K, Grosche J, Skatchkov SN, Schinkinger S, Foja C, Schild D, et al. Müller cells are living optical fibers in the vertebrate retina. Proceedings of the National Academy of Sciences. 2007;104(20):8287–8292.
View Article
Google Scholar

[122] View Article

[123] Google Scholar

[ref37] 37. Solovei I, Kreysing M, Lanctôt C, Kösem S, Peichl L, Cremer T, et al. Nuclear architecture of rod photoreceptor cells adapts to vision in mammalian evolution. Cell. 2009;137(2):356–368. pmid:19379699
View Article
PubMed/NCBI
Google Scholar

[125] View Article

[126] PubMed/NCBI

[127] Google Scholar

[ref38] 38. Kreysing M, Boyde L, Guck J, Chalut KJ. Physical insight into light scattering by photoreceptor cell nuclei. Optics letters. 2010;35(15):2639–2641. pmid:20680084
View Article
PubMed/NCBI
Google Scholar

[129] View Article

[130] PubMed/NCBI

[131] Google Scholar

[ref39] 39. Błaszczak Z, Kreysing M, Guck J. Direct observation of light focusing by single photoreceptor cell nuclei. Optics express. 2014;22(9):11043–11060. pmid:24921803
View Article
PubMed/NCBI
Google Scholar

[133] View Article

[134] PubMed/NCBI

[135] Google Scholar

Figures

Abstract

Author summary

Introduction

Design and implementation

Results

Light-sheet microscopy

Adaptive optics simulations

Discussion

Availability and future directions

Supporting information

S1 Text. 25 pages description of the numerical method, validation, benchmarks, details of the numerical experiments, and example listings.

S1 Video. Wave-optical simulation of the image-formation process in light-sheet microscopy.

S2 Video. Illustration of a single PSF calculation inside the tissue via the propagation of analytically defined diffraction-limited input fields.

S3 Video. A biobeam generated video illustrating rigorous wave-optical mimicry of a wide-field microscope.

S4 Video. Screencast of an interactive command line session demonstrating biobeam’s capabilities and speed.

S5 Video. Video showing the predefined illumination modes and simulated light sheets being scanned through a biologically plausible tissue model.

S6 Video. Showing the simulation of a aberration pre-compensated wavefront focusing deep into tissue and the shift-shift memory effect.

S7 Video. Example of a plane-by-plane illumination of a tissue model mimicking an embryo.

S1 Fig. Validation of biobeam with analytical solutions.

S2 Fig. Propagation of different predefined input fields through a tissue model of size (100μm, 100μm, 100μm) and grid dimension (10243).

S3 Fig. Detection aberration and PSF calculation.

S4 Fig. Calculating the aberrations of the detection PSF for a given z plane within a synthetic tissue model.

S5 Fig. Diffraction around a knife edge.

S6 Fig. Experimental validation on a commercial light-sheet microscope.

S7 Fig. Custom micro-projection setup built in our lab, controlled with custom LabView programs.

S8 Fig. Experimental micro-projection of a test-chart through a glass sphere and comparison with simulation.

S9 Fig. Simulation of shift-shift memory effect.

S10 Fig. Aberration correction.

S1 Table. Runtimes of different methods for the simulation of a plane wave propagating through a given refractive index distribution.

Acknowledgments

References

S2 Fig. Propagation of different predefined input fields through a tissue model of size (100μm, 100μm, 100μm) and grid dimension (1024³).