Capturing the entire three-dimensional (3D) volume in a single camera shot in microscopy sounds like a magic at first glance, but the original concept to achieve this in photography, called light field camera, was introduced early in 1908 by Gabriel Lippmann, who was also known for producing the first color photographic plate in the world. Due to the recent advancements in digital image sensors and image processing capabilities using computers, light field camera becomes more and more widely applied. In the meantime, introducing this century-old idea into microscopy, which is a different, but closely related field to photography, turned out to be very fruitful [1,2,3] because microscopists and biologists are desperately looking for volumetric imaging methods to capture living dynamics that are three-dimensional in nature.
We got interested in light field microscopy (LFM) by accident when we were working with Shack Hartmann wavefront sensor in adaptive optical microscope . Shack Hartmann wavefront sensor employs a micro-lens array conjugated to the back pupil of the imaging objective to infer the local wavefront gradient sampled at each micro-lens by measuring the positional shift of the image of a “guide star”. Because the fluorescent structures, which can serve as native “guide stars”, are very often three-dimensional in biological samples, we need to extend the capability of conventional Shack Hartmann sensor to deal with such situations. In the end, we realized that Shack Hartmann sensor actually captures projection views of the 3D object and could be used to reconstruct this 3D object when employing the principle of projection tomography. Later, we also realized that it was actually a different realization of the original concept of light field imaging. In 2017, we implemented this idea and developed eXtended field-of-view Light Field Microscopy (XLFM) . Due to its instantaneous volumetric imaging capability, we recorded the neural activity over the whole brain of a freely behaving larval zebrafish free of motion-blur. XLFM features extended imaging volume, faster reconstruction speed due to the definition of a spatially invariant Point Spread Function (PSF) and free of reconstruction artifacts. However, when we want to further improve the imaging performance of XLFM, we found that the lack of optical sectioning capability substantially limited the practically achievable resolution and sensitivity.
Introducing optical sectioning into LFM has motivated a great number of new ideas, including speckled illumination , light sheet illumination  and wide field temporal focusing two photon excitation . We would like to find a new way to achieve optical sectioning while preserving many important features that could be vital for its wide applications, including epi-illumination configuration for easier sample mounting and one-photon excitation for scalability in extending imaging volumes further. Borrowing the idea of confocal detection in confocal microscopy was naturally the first idea jumped into our mind due to its vast popularity in biological studies. However, after testing the spinning disk confocal imaging scheme in LFM, we realized that it was not feasible because all forms of conventional confocal microscopes were designed to collect signals from a single imaging plane at a time and were not compatible with LFM. If the conventional confocal detection scheme is not applicable, can we generalize the original concept to achieve optical sectioning over a volume instead of a thin plane? The key is to choose the right way to selectively excite fluorescent samples. If we employ tightly focused laser spot for fluorescence excitation, the fluorescent signal from different depth will mix and difficult to separate. It took us quite some time to figure out that we should simply make the excitation laser a loosely focused vertical beam with long beam waist to cover the focusing volume. In this way, the fluorescent signals excited along this beam at different depth will spread laterally on the camera under different micro-lens in the same way as the PSF we measured. Then, we could design different physical apertures for different micro-lenses to spatially block out-of-focus background. In the end, we chose to shape the excitation beam into a vertical light sheet instead of a beam to simplify the scanning mechanism to cover a volume. This new method is termed confocal LFM (CLFM).
Demonstrating interesting and important applications of developed technology is crucial to show its potential and gain wider applications. To this end, we talked with many biologists for ideas. We collaborated with Jiulin Du’s Lab to record calcium events over the whole larval zebrafish brain during its prey capture processes and demonstrated greatly improved imaging resolution and sensitivity comparing to our previous results. We also showcased the imaging of calcium signal in populations of neurons and circulating blood cells in large vascular networks in living mouse cortex. In the future, the scalability of CLFM could potentially allow even larger volume coverage and higher imaging throughput if larger cameras and imaging objectives are available.
- Levoy, M., Ng, R., Adams, A., Footer, M. J., & Horowitz, M. (2006). Light field microscopy. international conference on computer graphics and interactive techniques.
- Broxton, M., Grosenick, L., Yang, S., Cohen, N., Andalman, A. S., Deisseroth, K., & Levoy, M. (2013). Wave optics theory and 3-D deconvolution for the light field microscope. Optics Express, 21(21), 25418-25439.
- Prevedel, R., Yoon, Y., Hoffmann, M., Pak, N., Wetzstein, G., Kato, S., ... & Vaziri, A. (2014). Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nature Methods, 11(7), 727-730.
- Wang, K., Milkie, D. E., Saxena, A., Engerer, P., Misgeld, T., Bronner, M. E., ... & Betzig, E. (2014). Rapid adaptive optical recovery of optimal resolution over large volumes. Nature Methods, 11(6), 625-628.
- Cong, L., Wang, Z., Chai, Y., Hang, W., Shang, C., Yang, W., ... & Wen, Q. (2017). Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio). eLife 6:e28158.
- Taylor, M. A., Nöbauer, T., Pernia-Andrade, A., Schlumm, F., & Vaziri, A. (2018). Brain-wide 3D light-field imaging of neuronal activity with speckle-enhanced resolution. Optica, 5(4), 345–353.
- Wagner, N., Norlin, N., Gierten, J., De Medeiros, G., Balazs, B., Wittbrodt, J., ... & Prevedel, R. (2019). Instantaneous isotropic volumetric imaging of fast biological processes. Nature Methods, 16(6), 497-500.
- Hsu, F. C., Da Sie, Y., Lai, F. J., & Chen, S. J. (2018, February). Volumetric bioimaging based on light field microscopy with temporal focusing illumination. In Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XXV (Vol. 10499, p. 104991Q). International Society for Optics and Photonics.