Improving spatial resolution has always been an important goal in optical microscopy. A highly useful super-resolution technique is 3D SIM , invented by Mats Gustafsson. 3D SIM excites the sample with 3-beam interference, providing information outside the diffraction-limited passband that is encoded in the fluorescence captured by diffraction-limited images. Decoding this extra information mathematically yields a super-resolution reconstruction with doubled resolution in all three dimensions compared to wide-field microscopy. However, its axial resolution is still limited to ~300 nm, considerably worse than its ~120 nm lateral resolution. This leads to anisotropic reconstructions, in which fine features are distorted and badly blurred along the axial (z) dimension.
In follow up work, termed I5S , Mats and his lab member Lin Shao showed that axial resolution could be further improved using the interference pattern generated from two opposed objectives. The same objectives can be used to collect fluorescence, which is also interfered. The combination of coherent illumination and detection produces ~100 nm isotropic spatial resolution. However, the optical complexity and operational difficulty of I5S made it difficult to implement, and most subsequent improvements in SIM have focused on improving the lateral (x-y) resolution rather than the axial (z) resolution.
To address the problem of axial resolution, my supervisor Dr. Hari Shroff (former senior investigator at NIH/NIBIB, now senior group leader, HHMI/Janelia Research Campus), Patrick La Riviere (Professor at University of Chicago) and I began working on the problem in late 2018. In the beginning, Hari and I did not start with 3D SIM directly but rather attempted to improve the axial resolution of instant SIM  (which offers a lateral resolution of 145 nm and an axial resolution of 350 nm). Motivated by previous “standing-wave microscopes” , we attempted to use an oblique incident beam to provide axial illumination spatial frequencies that could in principle offer better axial resolution. However, our attempts failed as the interference pattern was never flat, but rather curved due to off-axis aberrations. These aberrations became more severe at higher NA, implying that we would be unable to obtain the desired interference pattern. After this setback, we focused our attention on 3D SIM. Since the illumination pattern in this system already contained a mixture of axial frequencies, we only needed to find a way to further expand the number of high frequency components.
When building our home-made 3D SIM system in 2020, we also came across conceptual work published by James Manton et al. , in which the central 3D SIM illumination beam is isolated by a second objective, re-imaged to a mirror, and reflected back towards the sample – thereby producing an interference pattern with higher axial illumination spatial frequencies. Hari and I were very excited by this development, as it suggested a straightforward path to improving the axial resolution in 3D SIM. However, notable challenges existed in the proposed design. First, two opposed objectives are still required as in I5S, increasing the complexity of alignment and requiring active feedback control of multiple optical elements. Second, the central beam passes through the interface of the immersion medium and air, the second objective, and a collimating lens twice, which inevitably introduces severe wavefront distortion. Moreover, such a long optical path design requires a laser with very high coherence length, ruling out common laser sources used in microscopy.
We wondered if there was an easier and more practical way to achieve 4-beam interference on an existing 3D SIM system. Using our previous experience with standing waves, we showed that placing a mirror directly opposite the sample enabled 4-beam interference with higher spatial frequency content than 3D SIM illumination. Although this was a powerfully simple concept, we still faced many problems in practice, such as determining how to mount the sample, reduce mechanical vibrations, compensate for drift of the stage and mirror, deal with multicolor imaging, and learn the reconstruction process for 3D SIM. We experienced many failures, but learned from them, in the end producing a useful 4-beam SIM which maintained the ~2-fold lateral resolution enhancement of 3D SIM over wide-field microscopy while offering ~2-fold better axial resolution than 3D SIM, i.e., offering near-isotropic imaging with ~120 nm lateral and 160 nm axial resolution. We then demonstrated that this resolution improvement provided crisp axial views of different samples, including 100-nm beads, bacterial membranes, the outer mitochondrial membrane, and dual-color samples (Figure 1).
Despite these successes, volumetric time-lapse (‘4D’) imaging with 4-beam SIM proved challenging, as we observed pronounced phototoxicity. This is unsurprising because imaging even a modestly sized cellular volume with 4-beam SIM requires ~1,000 raw images, which leads to photobleaching and phototoxicity. Given that 3D SIM introduces less dose than 4-beam SIM, is more robust to wavefront distortions, and can enable sustained 4D imaging , we considered computational strategies for improving the axial resolution of 3D SIM without introducing additional illumination dose. We decomposed the overall challenge into two sub-problems: 1. How to denoise 3D SIM images with low SNR? 2. How to improve the axial resolution of 3D SIM data?
In 2021, Dr. Jiji Chen (NIH/NIBIB) and collaborators from SVision demonstrated three-dimensional residual channel attention networks (3D RCAN)  as a powerful deep learning (DL) method for denoising 4D fluorescence microscopy images. However, although applying a 3D RCAN denoising model on raw 3D SIM data prior to reconstruction provided some benefit, the reconstructions still showed obvious patterned noise. We found that applying a second 3D RCAN denoising model provided much better reconstructions and adopted this two-step denoising procedure.
Recently, my colleague Dr. Yicong Wu (NIH/NIBIB) in the Shroff Lab proposed a DL strategy to improve lateral resolution in one direction by using improved lateral resolution in another direction , modifying an earlier method based on content-aware image restoration (CARE) networks . We realized that a similar approach could be applied to improve the axial resolution of 3D SIM. Considering that the lateral views always have better resolution than the axial views, we synthetically blurred one axial view along its lateral axis to resemble its axial axis and trained a neural network to reverse this blur. We then obtained a prediction with near isotropic resolution by applying the trained network to six digitally rotated views, using techniques from tomography to combine the different views. We also verified that another DL method, Richardson-Lucy Network (RLN), could also be used to improve axial resolution  (see our previous Blog post: Incorporating the image formation process into deep learning improves network performance | Nature Portfolio Bioengineering Community) Interestingly, we found that using the original CARE strategy  did not produce good reconstructions, possibly because axial specimen views looked quite different than the lateral specimen views the network was trained on. Therefore, we directly incorporated axial views into the training procedure. We used our 4-beam SIM method as a check on the DL predictions, verifying that the two methods provided similar results (Figure 2a-c).
Our two-step denoising pipeline produced high-quality 3D SIM predictions that could then be input to our axial resolution enhancement method, producing high SNR reconstructions with ~120 nm isotropic resolution from low SNR input. As we were able to turn the illumination intensity down significantly, this pipeline enabled 4D isotropic super-resolution imaging on different biological samples (Figure 2d-f).
In conclusion, our work  demonstrates two distinct methods to improve axial resolution in 3D SIM with minimal or no modification to the underlying 3D SIM optical system. We look forward to developing more new technologies to improve the resolution of microscopy, particularly in thicker samples.
- Gustafsson, M. G. L. et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J. 94, 4957–4970 (2008).
- Shao, L. et al. I5S: wide-field light microscopy with 100-nm-scale resolution in three dimensions. Biophys. J. 94, 4971–4983 (2008).
- York, A. G. et al. Instant super-resolution imaging in live cells and embryos via analog image processing. Nat. Methods 10, 1122–1126 (2013).
- Bailey, B., Farkas, D. L., Taylor, D. L. & Lanni, F. Enhancement of axial resolution in fluorescence microscopy by standing-wave excitation. Nature 366, 44–48 (1993).
- Manton, J. D., Strohl, F., Fiolka, R., Kaminski, C. F. & Rees, E. J. Concepts for structured illumination microscopy with extended axial resolution through mirrored illumination. Biomed. Opt. Express 11, 2098–2108 (2020).
- Shao, L., Kner, P., Rego, E. H. & Gustafsson, M. G. L. Super-resolution 3D microscopy of live whole cells using structured illumination. Nat. Methods 8, 1044–1046 (2011).
- Chen, J. et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat. Methods 18, 678–687 (2021).
- Wu, Y. et al. Multiview confocal super-resolution microscopy. Nature 600, 279–284 (2021).
- Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
- Li, Y. et al. Incorporating the image formation into deep learning improves network performance. Nat. Methods 19, 1427–1437 (2022).
- Li, X., Wu, Y., Su, Y. et al. Three-dimensional structured illumination microscopy with enhanced axial resolution. Nat. Biotechnol. (2023). https://doi.org/10.1038/s41587-022-01651-1
Please sign in or register for FREE
If you are a registered user on Nature Portfolio Bioengineering Community, please sign in