Shouyu
Wang
*abc,
Huachuan
Huang
d,
Aihui
Sun
e,
Lin
Zhu
be,
Wei
Guo
ab,
Keding
Yan
f and
Liang
Xue
g
aJiangsu Province Engineering Research Center of Integrated Circuit Reliability Technology and Testing System, Wuxi University, Wuxi, Jiangsu 214105, China. E-mail: shouyu29@cwxu.edu.cn
bOptiX+ Laboratory, School of Electronics and Information Engineering, Wuxi University, Wuxi, Jiangsu 214105, China
cSingle Molecule Nanometry Laboratory (Sinmolab), Nanjing, Jiangsu, China
dSchool of Manufacture Science and Engineering, Key Laboratory of Testing Technology for Manufacturing Process, Ministry of Education, Southwest University of Science and Technology, Mianyang, Sichuan 621010, China
eComputational Optics Laboratory, Jiangnan University, Wuxi, Jiangsu 214122, China
fAdvanced Institute of Micro-Nano Intelligent Sensing (AIMNIS), School of Electronic Information Engineering, Xi'an Technological University, Xi'an, Shaanxi 710032, China
gCollege of Electronics and Information Engineering, Shanghai University of Electric Power, Shanghai, 200090, China
First published on 14th February 2024
Because phase has significantly higher contrast than amplitude, particularly for label-free specimens, and provides a new perspective for morphology and shape testing, quantitative phase microscopy has become an effective means in optical imaging and testing. We designed dual-view transport of intensity phase imaging, which comprehensively considers real-time imaging, simple configuration, and fast phase retrieval. This technique employs two imaging recorders to simultaneously capture under- and over-focus images and recovers the quantitative phase distributions from these defocused images by solving the Poisson equation. Based on such an idea, we designed PhaseRMiC as a phase real-time microscopy camera and PhaseStation as a compact phase imaging work station, and they have been successfully used in live cell observation, whole-slide imaging, and flow cytometry for various purposes, such as specimen detection, counting, recognition, and differentiation. In this work, we first briefly introduce the principle of the dual-view transport of intensity phase imaging, next provide the details of our designed PhaseRMiC and PhaseStation, and finally demonstrate their applications in real-time, field of view scanning, and microfluidic imaging, respectively. Besides, we also compare PhaseRMiC and PhaseStation with other quantitative phase microscopy equipment and lay out their prospects for future applications. We believe our designed dual-view transport of intensity phase imaging as well as its derivatives, PhaseRMiC and PhaseStation, can be a promising choice for quantitative phase microscopy.
Various quantitative phase microscopy techniques have been designed. The most traditional quantitative phase microscopy technique relies on interference, like interferometry or holography.12,13 The phase information is coded in the fringes generated via sample and reference wave interference. Therefore, the sample phase can be demodulated from fringe patterns. Various phase retrieval algorithms have been designed, mainly according to spatial and spectral demodulations. Spatial demodulation, or phase-shifting demodulation, can extract phase from a series of phase-shifting fringe patterns with extremely high accuracy.14,15 While spectral demodulation can reconstruct phase from a single-shot fringe pattern with the carrier to pursue real-time imaging.16–18 The most successful application of interference-based quantitative phase imaging is optical shop testing.19 Combined with microscopy, it is not only used in micro-structure detection (such as MEMS) but is also employed in biomedical applications such as label-free tissue and cell imaging.9,11,20 Unfortunately, due to the requirement for dual-wave interference, interference-based quantitative phase imaging often relies on complicated and bulky systems. Developed from coherent diffraction imaging, ptychography can provide phase imaging with high accuracy and a large field of view (FoV) through scanning in hardware and iterative phase retrieval in software.21–26 Ptychography in the spatial domain, ptychographic iterative engine, reconstructs phase from a series of diffraction patterns obtained by scanning sample or probe light in space with overlapping in neighboring scanning.21–23 Besides, ptychography in the spectral domain, Fourier ptychography, retrieves phase from a series of images taken at different angles of illumination, which carry the information in different spectra with overlapping in the neighboring illumination.24–26 Ptychography can achieve quantitative phase imaging with high accuracy and a large FoV as an advantage, but it has the disadvantage of long-time data collection and phase reconstruction. Different from interferometry/holography and ptychography, which mostly require coherent light, differential phase contrast imaging, relying on spatially partially coherent light, can retrieve specimen phase from a series of images with asymmetric patterns in different illuminations or pupils.27–29 Therefore, it has better immunity to speckle noise, a simpler system, and a faster processing speed compared to interferometry/holography and ptychography. But it still needs multiple shots for phase retrieval. Also relying on partially coherent light, transport of intensity phase imaging can retrieve specimen phase from a series of multi-focus images by solving the Poisson equation.30–32 It shares the same advantages of differential phase contrast imaging as speckle immunity, a simple system, and faster processing speed, and suffers from multiple shots for phase retrieval. Besides these quantitative phase imaging techniques, there are still other methods, such as Shack–Hartmann sensing33 and coded aperture phase imaging.34–37 However, Shack–Hartmann sensing suffers from low spatial resolution, so it is widely used for wavefront sensing. Coded aperture phase imaging has a simple system but requires multiple shots and long-time phase retrieval. Therefore, interferometry/holography, ptychography, differential phase contrast imaging, and transport of intensity phase imaging are the mainstream techniques in quantitative phase microscopy.
To suit various kinds of applications, quantitative phase imaging techniques with simple systems, dynamic imaging capability, and fast processing speeds are preferred. Interferometry/holography supports dynamic imaging, but many of them rely on complicated and bulky systems. Many updated strategies have been reported to address the issue. For example, diffraction phase imaging integrates a grating and a pinhole to construct the single-path sample and reference beam configurations and reduce the system size.38 Quadriwave lateral shearing interferometry employs a chessboard grating instead of independent sample and reference systems to obtain shearing fringes and reduce the system size.39,40 Lensless digital holography, often based on in-line holography, can even miniaturize the system into a handheld size.41,42 But interferometry/holography still suffers from long-time phase unwrapping. Though coherent modulated imaging provides a way for single-shot quantitative phase imaging,43 ptychography is not a preferred solution for dynamic imaging due to its time-consuming iteration for phase retrieval. Differential phase contrast imaging has a relatively simple system and a fast-processing speed, and asymmetric patterns in different illuminations or pupils can be obtained simultaneously by relying on multi-wavelength illumination.44–47 But it cannot deal with dispersed specimens. Similarly, though relying on chromatic dispersion to simultaneously obtain multi-focus images is also a solution for real-time transport of intensity phase imaging,48 it cannot deal with dispersed specimens. Besides, using prism,49–53 grating,54–56 diffractive optical element,57 optical sieve,58,59 metasurface,60 and spatial light modulator,61 multi-focus images can also be simultaneously obtained; however, these methods suffer from limited FoVs and require complicated setups. To solve the problem, inspired by our previous works,62–64 we designed dual-view transport of intensity phase imaging,65–69 which comprehensively considers real-time imaging, simple configuration, and fast phase retrieval, as shown in Fig. 1(A). This technique employs two imaging recorders to simultaneously capture under- and over-focus images and recover the quantitative phase distributions. Additionally, based on the idea, we designed PhaseRMiC as a phase real-time microscopy camera and PhaseStation as a compact phase imaging work station, as revealed in Fig. 1(B) and (C), and they have been successfully used in live cell observation, whole-slide imaging, and flow cytometry for various purposes, such as specimen detection, counting, recognition, and differentiation.
![]() | ||
Fig. 1 Dual-view transport of intensity phase imaging and devices. (A) Dual-view transport of intensity phase imaging. Reproduced from ref. 64 with permission from The Optical Society, copyright 2019; (B) PhaseRMiC. Reproduced from ref. 65 with permission from The Optical Society, copyright 2021; (C) PhaseStation. Reproduced from ref. 68 with permission from The Optical Society, copyright 2023. |
In this work, we introduce the dual-view transport of intensity phase imaging and demonstrate its derivatives, PhaseRMiC and PhaseStation, in applications. In detail, we introduce the principle of the dual-view transport of intensity phase imaging and provide its algorithms, including FoV correction and phase retrieval, in section 2. Then, we propose the scheme for the dual-view transport of intensity phase imaging and the configurations of the PhaseRMiC and PhaseStation in section 3. Subsequently, we demonstrate their various applications in dynamic imaging for live cell observation, whole slide imaging for target recognition and detection, and microfluidic imaging for flow cytometry in section 4. Finally, we compare PhaseRMiC and PhaseStation with other quantitative phase imaging equipment and point out the pros and cons of the dual-view transport of intensity phase imaging in section 5. Additionally, we also mention the future development directions of the PhaseRMiC and PhaseStation in the same section. We hope that this work will serve as a helpful resource for quantitative phase imaging.
Like interferometry and coherent diffraction imaging, which code phase into fringe and diffraction pattern, respectively, transport of intensity phase imaging code phase into imaging intensity. In other words, a phase change distorts the wavefront and thus generates an inhomogeneity in the intensity distribution. From the phase-distorted inhomogeneous intensity, the phase can be reconstructed by solving the transport of intensity equation, as demonstrated in eqn (1). The theoretical background of the transport of intensity equation was first discussed84,85 from the classical Helmholtz equation. The derivative can be found in ref. 86 and 87 in detail. In eqn (1), φ is the phase distribution, λ is the wavelength, k as 2π/λ is the wave number, Δ is the defocus distance, and ∇⊥ is the lateral gradient operator.
![]() | (1) |
Various methods have been reported to solve the transport of intensity equation,88–93 and the most classical one is based on the auxiliary function proposed by Teague84 and the derivative feature of the Fourier transform. In brief, the auxiliary function ϕ is defined as eqn (2), and the transport of intensity equation can be derived into eqn (3) and (4).
∇⊥ϕ = I∇⊥φ | (2) |
∇⊥2ϕ = −k∂zI | (3) |
∇⊥2φ = ∇⊥·(I−1∇⊥ϕ) | (4) |
According to the two-dimensional derivative feature of the Fourier transform (), the auxiliary function can be extracted according to eqn (5), and the phase under detection can finally be retrieved according to eqn (6).
![]() | (5) |
![]() | (6) |
∂zI can be usually computed using center finite difference approach from under-, in- and over-focus images. But multi-distance approaches can obtain higher phase reconstruction accuracy but require more image captures.94–99 The tradeoff between phase retrieval accuracy and multi-focus images should be comprehensively considered in different applications.86,100,101 For example, multi-distance approaches are preferred to pursue high accuracy in phase reconstruction, and fewer multi-focus image captures can accelerate image recording efficiency and provide a solution for real-time quantitative phase imaging. Specifically, our proposed dual-view transport of intensity phase imaging techniques and devices can simultaneously capture under- and over-focus images, thus supporting real-time quantitative phase imaging.
![]() | (7) |
![]() | (8) |
In the following, numerical simulation is employed to prove the effectiveness of the dual-view transport of intensity phase imaging.
According to the above numerical simulation, it is proven that the dual-view transport of intensity phase imaging can support quantitative phase imaging, and additionally, it only requires two image recordings, simplifying the optical system and supporting real-time imaging. Therefore, it is a promising way to develop devices for quantitative phase imaging. The following two devices, PhaseRMiC and PhaseStation, which have been developed and employed in many applications, are introduced in the next section.
To solve these problems, we proposed an updated PhaseRMiC device, as shown in Fig. 3(A),65 which is an image recorder that is linked to a commercial microscope. Its configuration is very simple, consisting only of a prism (Daheng Optics, China) and a board-level camera (Daheng Image, China) with two CMOS imaging sensors integrated using 3-D printed structures, as shown in Fig. 3(B)–(D). In detail, the wave carrying the specimen phase from the C-mount of the commercial microscope is first split by a prism and then reaches two CMOS imaging sensors. Additionally, because the interval between these two CMOS imaging sensors and the prism was different, these two images were captured in different focal planes. By adjusting the sample position, these imaging sensors can capture the under- and over-focus images for further phase retrieval.
![]() | ||
Fig. 3 PhaseRMiC hardware. (A) Prototype. Reproduced from ref. 65 with permission from The Optical Society, copyright 2021; (B) configuration; (C) inner structure; (D) optical system. |
Different from PhaseRMiC, which only works as a phase imaging camera, PhaseStation combines microscopy and quantitative phase imaging, so it works as a quantitative phase microscope. Fig. 4(A) depicts the PhaseStation's prototype, and Fig. 4(B) reveals its scheme. The illumination was from an LED source filtered using an interference-based filter and collimated using two lenses. The almost parallel wavefront was incident on the sample, which was fixed on the sample stage, and the transmitted wavefront was collected and magnified using a micro-objective, then split by the prism, and finally imaged by two image sensors located in different imaging planes, as revealed in Fig. 4(C). By adjusting the sample position, these imaging sensors can capture the under- and over-focus images for further phase retrieval.
![]() | ||
Fig. 4 PhaseStation hardware. (A) PhaseStation prototype; (B) PhaseStation scheme; (C) dual-view imaging part. Reproduced from ref. 68 with permission from The Optical Society, copyright 2023. |
The defocus distance between two imaging sensors was precisely measured by laser ranging. After fixing the laser ranger and prism-imaging sensor module, these two imaging sensors were measured in sequence; that is, one was detected while the other was blocked. The interval between these two imaging sensors was the difference between the two measured distances.
A standard target (grid plate) was used for FoV calibration, and the in-focus plane in the specimen space could be determined using the classic Brenner gradient in-focus criterion by laterally shifting the precision translation specimen stage. Afterward, the standard sample was shifted to the central positions of the in-focus planes, which corresponded to two image recorders, and two under- and over-focus images could be captured. Relying on the phase correlation-based FoV correction method, all the scale, rotation, and translation differences between the captured under- and over-focus images could be determined and compensated. For example, two images, i1 and i2, with a translation of Δx and Δy are illustrated in eqn (9).
i2(x, y) = i1(x − Δx, y − Δy) | (9) |
I2(u, v) = I1(u, v)e−j2π(uΔx+vΔy) | (10) |
![]() | (11) |
Besides translation, scale and rotation also exist in the under- and over-focus images. In the phase correlation-based FoV correction, scale and rotation are first determined and compensated, and translation is next determined and compensated. In detail, two images i and i′ with scale (s), rotation (θ), and translation (x0, y0) are listed in eqn (9), in which x′ = s·x·cosθ + s·y·sin
θ − x0 and y′ = −s·x·sin
θ + s·y·cos
θ − y0 describe the relation between (x, y) and (x′, y′).
i(x, y) = i′(x′, y′) | (12) |
![]() | (13) |
![]() | (14) |
Taking the logarithm of r/s to convert the division into subtraction, logs and θ in eqn (15), which are equivalent to Δx and Δy in eqn (9), can be determined.
![]() | (15) |
Before using PhaseRMiC and PhaseStation for quantitative phase imaging, the specimen under detection should first be set at the in-focus plane. In other words, it should be guaranteed that two image recorders can capture under- and over-focus images, respectively. With the prior FoV calibration, the mismatch between the two image sensors can be precisely determined. As a result, by first compensating the FoV and then computing the average, the average intensity of those captured by two image recorders can be quickly obtained while laterally shifting the sample. This computed average intensity is evaluated by an in-focus criterion (such as the classic Brenner gradient (B) in eqn (16), where s(i,j) represents pixel intensity at the pixel position (i,j) and m was chosen as 2), and the specimen position can be determined. In other words, it is the autofocus process of PhaseRMiC and PhaseStation.
![]() | (16) |
To summary the phase retrieval process, Fig. 5 demonstrate the flowchart of the dual-view transport of intensity phase imaging.
![]() | ||
Fig. 7 Dynamic imaging applications. Time-series images of under- and over-focus as well as the reconstructed phase distributions of Vero cells during trypsinization in 180 s. Reproduced from ref. 65 with permission from The Optical Society, copyright 2021. |
Due to their excellent single-shot quantitative phase imaging capability and their convenience in applications, these dual-view transport of intensity phase imaging-based devices can be used in the future in many label-free dynamic observation applications.
![]() | ||
Fig. 8 Whole-slide imaging applications. (A) Under- and over-focus images in different sub-FoVs during scanning; (B) FoV-stitched under- and over-focus images; (C) quantitative phase distribution reconstructed from (B); (D) FoIs; (E) BSRNet; (F) blood cell recognition. Reproduced from ref. 67 with permission from Elsevier Sciences, copyright 2022. |
Similarly, we also employed the same system to detect the bubbles in the transformer oil to evaluate its quality since bubbles are generated during transformer running.66 The transformer oil sample was first introduced into the specimen chamber; next, the specimen chamber was scanned to record under- and over-focus images in the extended FoVs; and finally, bubble volume can be quantified according to the bubble phase distributions. The quality of the transformer oil (new, available, and unavailable) can be determined according to the oil-to-gas volume ratio.
These dual-view transport of intensity phase imaging-based devices could be prototypes of the whole-slide quantitative phase imaging device, which may provide a new perspective for digital pathology.
![]() | ||
Fig. 9 Microfluidic imaging applications. (A) Microfluidic chip; (B) phase image of a frame and cell recognition; (C) statistical analysis; (D) single-cell rotation in the x–y plane; (E) single-cell rotation in the y–z plane; (F) dual-cell binary rotation in the y–z plane. Reproduced from ref. 69 with permission from The Optical Society, copyright 2023. |
We used the phase imaging flow cytometry system for cell imaging and analysis. The C6 cell solution at a concentration of 2 × 105 cells per mL was injected into a microfluidic channel with a flow rate of 36 μL min−1, and phase images of a fixed FoV but with flowing C6 cells were reconstructed, such as one shown in Fig. 9(B). According to threshold segmentation, C6 cells could be recognized in Fig. 9(C). Besides, its morphological parameters, such as cellular diameter and area, can be quantified. With multiple frames, statistical cellular parameters can be obtained and analyzed. Besides high-throughput cellular imaging, this phase imaging flow cytometry system can also be applied for monitoring cell dynamics in the flow. For example, according to the serial phase images of cells flowing across the FoV, single-cell rotation in the x–y plane, in the y–z plane, and dual-cell binary rotation in the y–z plane can be distinguished, as revealed in Fig. 9(D), (E) and (F), respectively.
Relying on the PhaseRMiC, the phase imaging flow cytometry system has the advantages of simple configuration, accurate phase retrieval, and real-time imaging; therefore, dual-view transport of intensity phase imaging is a promising tool for developing phase imaging flow cytometry.
Phase retrieval methods | Phase retrieval performance | ||||||
---|---|---|---|---|---|---|---|
Categories | Accuracy | Dynamic | Speed | Complexity | Field of view | Resolution | |
Interferometry/holography | Spatial demodulation/on-axis | Very high | Multi-shots | Slow | Complex | Normal | High |
Gold standard | Phase shifting | Unwrapping | Phase shifting | Depend on imaging system | Depend on imaging system | ||
Spectral demodulation/off-axis | High | Single-shot | Slow | Complex | Normal | High | |
Carrier frequency | Unwrapping | Often non-common path | Depend on imaging system | Depend on imaging system | |||
Quadriwave lateral shearing interferometry/shearing | High | Single-shot | Normal | Simple | Normal | High | |
Carrier frequency | Bidirectional processing | Common path | Depend on imaging system | Depend on imaging system | |||
Ptychography | Ptychographic iterative engine | Very high | Multi-shots | Very slow | Normal | Very large | Very high |
FoV scanning | Iteration | Lensfree but scanning required | FoV scanning | Diffraction limit | |||
Fourier Ptychography | Very high | Multi-shots | Very slow | Complex | Large | Very high | |
Spectral scanning | Iteration | Scanning required | Break through space bandwidth product | Break through space bandwidth product | |||
Coherent modulated imaging | High | Single-shot | Very slow | Simple | Very small | High | |
Single modulator | Iteration | Lens and scanning free | Limited by CMI principle | Depend on imaging system | |||
Differential phase contrast imaging | Classical | High | Multi-shots | Fast | Complex | Normal | High |
Illumination/pupil scanning | Unwrapping free | Illumination/pupil scanning | Depend on imaging system | Depend on imaging system | |||
Color coding | Normal | Single-shot | Fast | Normal | Normal | High | |
Dispersion | Simultaneous imaging with different illuminations/pupils | Unwrapping free | Color coding | Depend on imaging system | Depend on imaging system | ||
Transport of intensity phase imaging | Classical | High | Multi-shots | Fast | Complex | Normal | High |
Focal scanning | Unwrapping free | Focal scanning | Depend on imaging system | Depend on imaging system | |||
Achromatic | Normal | Single-shot | Fast | Simple | Normal | High | |
Dispersion | Simultaneous multi-focal imaging | Unwrapping free | Multi-wavelength sources | Depend on imaging system | Depend on imaging system | ||
FoV splitting | High | Single-shot | Fast | Normal | Small | High | |
Simultaneous multi-focal imaging | Unwrapping free | Extra elements | FoV splitting | Depend on imaging system | |||
Dual-view | High | Single-shot | Fast | Simple | Normal | High | |
Proposed | Simultaneous multi-focal imaging | Unwrapping free | Dual cameras and prism | Depend on imaging system | Depend on imaging system |
However, besides the advantages of the transport of intensity phase imaging and its dual-view versions, they still suffer from a series of problems. Firstly, its accuracy is limited, especially compared to ptychography and interferometry/holography. The main reason is low-frequency spatial noise due to the noise-resolution tradeoff in the transport of intensity phase imaging.86,100,101 Various approaches have been designed to alleviate the problem and thus improve phase retrieval accuracy, such as multi-distance approaches,91,94–99 computational illumination approaches,100–102 and iterative approaches.103,104 Unfortunately, multi-distance approaches can hardly support single-shot quantitative phase imaging because they need many defocus image captures, and iterative approaches consume a long time in phase retrieval. Besides, computational illumination approaches require heavy modifications to the commercial microscope. Though our designed dual-view transport of intensity phase imaging techniques and devices sacrifice the phase retrieval accuracy, they have the advantages of simple optical systems, fast phase retrieval, and real-time phase imaging capability. Therefore, they are preferred tools in dynamic imaging, whole-slide imaging, and microfluidic imaging. Secondly, the dual-view transport of intensity phase imaging techniques and devices depend on two cameras. Though they maintain relatively large FoVs and simple optical systems, they rely on the good consistency of two cameras. Therefore, intensity calibration and FoV compensation are required to improve phase retrieval accuracy. Besides the mentioned dual-camera tactic, a series of single-camera-based systems49–61 have also been reported to simultaneously capture multi-focus images for real-time quantitative phase imaging. These single-camera-based systems avoid the non-consistency problem, but they suffer from limited FoVs and require complicated and expensive setups.
Finally, it should be noted that quantitative phase imaging provides the global background of the specimen under detection. It is an updated version of bright/dark-field and phase contrast imaging since it provides not only high imaging contrast but also quantitative specimen information. Unfortunately, quantitative phase imaging loses the specificity that can provide more biological and chemical information on the specimen under detection. Therefore, to reflect more information, a combination of quantitative phase imaging and specific imaging (such as fluorescence imaging) as multi-modal imaging is a promising direction for various fields.
This journal is © The Royal Society of Chemistry 2024 |