This research represents a progressive step towards addressing the central question we have pursued for over a decade: Is it feasible to completely eliminate structured illumination microscopy (SIM) reconstruction artifacts while improving spatiotemporal resolution and maintaining the integrity of imaged structures? As a computational technique, SIM relies on the reconstruction phase to generate super-resolved (SR) images that are prone to artifacts. In 2018, we highlighted the susceptibility of SIM to intrinsic artifacts caused by Wiener inverse filter-induced random noise amplification, thus proposed to use spatiotemporal continuity to mitigate these artifacts1. In 2021, we further advanced the spatiotemporal capabilities of live-cell SR imaging by developing the sparse deconvolution algorithm utilizing the sparsity of the fluorescence images as another universal prior knowledge (for more information, refer to our previous blog post at https://bioengineeringcommunity.nature.com/posts/sparse-deconvolution-one-decisive-step-into-computational-fluorescence-superresolution). Using Sparse SIM, we have successfully attained a sub-70 nm resolution at a sampling rate of 564 Hz. This has allowed us to observe variously sized nuclear pores within the nuclear membrane and the dynamic behavior of complex actin networks and structures in living cells2.
Nonetheless, it was through significantly enhancing the resolution and contrast that we discovered the insufficiency of spatiotemporal continuity prior knowledge in suppressing certain types of artifacts. For example, numerous SR images obtained using Wiener-SIM appeared acceptable; however, periodic artifacts occasionally surfaced when the sparse deconvolution algorithm was applied to enhance the resolution further. This led to some users asserting that sparse deconvolution occasionally "induces" artifacts. In reality, this phenomenon can be ascribed to the combined relevance of spatiotemporal continuity and sparsity prior knowledge for periodic artifacts. These artifacts were not entirely eradicated by the Wiener reconstruction, and subsequently amplified through the sparse deconvolution process.
Upon examining cells displaying significant periodic artifacts, we discovered that these artifacts were generated due to high or non-homogeneous background fluorescence. This issue arises from the improper migration of low-frequency background signals to the high-frequency spectrum, resulting in periodic artifacts during SIM reconstruction. Two approaches have been suggested to address this problem: 1) directly removing background fluorescence from raw images or 2) employing a notch filter to eliminate the erroneous signal in the frequency domain. However, both methods come with their challenges. Removing background fluorescence may inadvertently eliminate weak reconstructed signals. At the same time, the notch filter approach (as used in NF-SIM3, 4/HiFi-SIM5) necessitates empirical parameter adjustments for different organelle images, even when captured under identical imaging parameters. Furthermore, both techniques can introduce non-linear intensity changes post-reconstruction, complicating the interpretation of fluorescent signals.
Hence, we propose that a quantitative and high-fidelity live-cell SR imaging method should meet the following four criteria:
- Transparency and repeatability: This ensures that different researchers achieve the same results using identical parameters.
- Minimal (or no) reconstruction artifacts and errors: This guarantees the accuracy and reliability of the reconstructed structures.
- A balance between spatial resolution and the integrity of fluorescent structures: An effective SR method should offer high resolution while preserving weak signals.
- Reconstruction should maintain the linearity of the processed fluorescence signal: This emphasizes the importance of accurate fluorescence signal interpretation.
Considering these criteria, we decide to adopt a physical-model-based approach to remove the background6. Given that images captured by the camera is the summation of fluorescence excited at different sample depths convolved with the corresponding point spread function, we wonder whether we can separate the convolved in-focal and out-of-focal emission (Fig. 1). Using the fluorescent subject within the focal plane (o(r, Zin) to approximate that of the out-of-focal plane (O(r, Zout), we obtained a new image formation model of SIM
in which I(r) is the sinusoidal intensity pattern for illumination, and h(r, Zin) and h(r, Zout) are the PSFs within the range of in-focus and out-of-focus. Thus, we can remove the background signal without reducing the genuine signals.
Fig. 1. The illustration depicts the integration of fluorescence emission from various layers (left) and the simulated point spread function (PSF) profile for both in-focus and out-of-focus conditions along the axial axis.
By incorporating the new background filtering (BF) method, BF-SIM effectively eliminates erroneous spots in the frequency spectrum and reduces periodic artifacts in both Wiener-SIM and Sparse SIM images (Fig. 2). Moreover, BF-SIM surpasses Wiener-SIM and HiFi-SIM in terms of performance, producing SR actin filaments with enhanced contrasts. The method also reliably detects weak actin branches, as evidenced by time-lapse imaging (Fig. 2). Overall, the BF-SIM reconstruction exhibited a level of linearity comparable to Wiener-SIM and superior to HiFi-SIM (Fig. 3). Finally, we employed densely labeled actin filaments and meshes to compare the performance of BF-Sparse-SIM with Sparse-SIM, which utilized Wavelets to eliminate background (Wavelets-Sparse-SIM) (Fig. 4). Although both techniques achieved approximately sub-70 nm resolution, the variance in fluorescence intensity fluctuations along actin filaments for BF-Sparse-SIM was only half to a quarter of that for Wavelets-Sparse-SIM, indicating that weaker signals are more continuous. In concordance with detecting more comprehensive structures, the density and average length of actin filaments measured by BF-Sparse-SIM increased by 1.5 to 2.5 times.
Fig. 2. The BF preprocessing enables the removal of erroneous spots in the frequency spectrum, suppresses fixed pattern artifacts, and retains weak signals.
Fig. 3. Lipid droplets of different fluorescent intensities reconstructed under the Wiener-SIM, HiFi-SIM, and BF-SIM.
Fig. 4. BF-Sparse-SIM provided actin filaments and meshes with higher fidelity and more complete structures than Waveletes-Sparse-SIM.
In summary, here we have proposed a physical-model-based background filtering method, which minimizes artifacts and ensures signal linearity and structural completeness as we push the resolution limit with the sparse deconvolution. Interestingly, this research also tackles a fundamental question in fluorescence imaging - how can one determine which background processing method is more accurate and closer to the ground truth, given that distinct methods may lead to varying fluorescence values?7 This implies that the proposed method can also be utilized for processing other wide-field fluorescence microscopy images. However, current methods to eliminate artifacts in SIM microscopy are not enough. It is already known that different organelles possess distinct reflective indices compared to the cytosol8, resulting in local distortions of pattern illumination and subsequent artifacts at various planes9. In the future, researchers must continue striving to completely eliminate SIM reconstruction artifacts while enhancing spatiotemporal resolution and preserving the integrity of imaged structures. This is because, in the end, as Carly Fiorina nicely stated, “The goal (of SR imaging ) is to turn data into information, and information into insight.”
- Huang, X. et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy. Nature biotechnology 36, 451-459 (2018).
- Zhao, W. et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy. Nature biotechnology 40, 606-617 (2022).
- Muller, M., Monkemoller, V., Hennig, S., Hubner, W. & Huser, T. Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ. Nature communications 7, 10980 (2016).
- Wicker, K., Mandula, O., Best, G., Fiolka, R. & Heintzmann, R. Phase optimisation for structured illumination microscopy. Optics express 21, 2032-2049 (2013).
- Wen, G. et al. High-fidelity structured illumination microscopy by point-spread-function engineering. Light, science & applications 10, 70 (2021).
- Mo, Y. et al. Quantitative structured illumination microscopy via a physical model-based background filtering algorithm reveals actin dynamics. Nature communications 14, 3089 (2023).
- Aaron, J. & Chew, T.L. A guide to accurate reporting in digital image processing - can anyone reproduce your quantitative analysis? Journal of cell science 134 (2021).
- Dong, D. et al. Super-resolution fluorescence-assisted diffraction computational tomography reveals the three-dimensional landscape of the cellular organelle interactome. Light, science & applications 9, 11 (2020).
- Mo, Y., Feng, F., Mao, H., Fan, J. & Chen, L. Structured illumination microscopy artefacts caused by illumination scattering. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences 379, 20200153 (2021).