Review : imaging technologies for flow cytometry

High-throughput single cell imaging is a critical enabling and driving technology in molecular and cellular biology, biotechnology, medicine and related areas. Imaging flow cytometry combines the single-cell imaging capabilities of microscopy with the high-throughput capabilities of conventional flow cytometry. Recent advances in imaging flow cytometry are remarkably revolutionizing single-cell analysis. This article describes recent imaging flow cytometry technologies and their challenges.


Introduction
Flow cytometry is a widespread and powerful technique employed in cell counting, biomarker detection and cell sorting.After the Coulter Principle was discovered in 1953 and Fulwyler applied this principle to sort cells in 1965, optical detections were soon adopted by flow cytometry and fluorescence-activated cell sorter (FACS) systems since the late 1960s. 1,24][5][6] At the same time, because of flow cytometry's successful applications in immunology and thanks to the growing appreciation of researchers for the complexity of the immune system, further technological advances have focused on satisfying the increasing need for polychromatic approaches to flow cytometry; researchers have developed flow cytometry to simultaneously measure 19 parameters-17 fluorescence and 2 scatter parameters in a high-speed manner. 7The broadly useful technology-flow cytometry-has been evolving slowly until imaging flow cytometry (IFC) became a resurgence of interest in the past decade. 8,94][15] Imaging is indubitably indispensable for cell analysis because images effectively convey certain messages about cells, such as cell size, shape, morphology, and distribution or location of labeled biomolecules within cells.7][18] Increasing flow cytometry's spatial resolution provides high-speed comprehensive analysis and indepth imagery of every individual cell.In addition, some erroneous results yielded in conventional flow cytometry can be eliminated by acquiring and analyzing the cell images to, for example, distinguish between cells, debris, and clusters of cells.
In contrast to pure quantitative measurements provided by conventional flow cytometry, a technique invented in the seventeenth century, microscopy allows capturing cell images that contain a wealth of information about a cell.While conventional flow cytometry measures forward scattered light to estimate the relative cell size, microscopy yields the exact cell size through its bright field image.Advances in the microscopy technique have realized both 3-dimensional and super resolutions that allow imaging of the biological structure and function beyond the diffraction limit, producing extraordinarily detailed fluorescent cell images, but it can take as long as several minutes to produce such high-content images.Automated microscopy coupled with automated image analysis can be relatively fast with a typical throughput of several hundreds of cells per second.0][21][22][23][24] Although LSC and high-throughput microscopy are suitable for studying complex biological pathways in a time-lapse manner, they are designed to work only with adherent cells.There are also some bottlenecks for high-throughput microscopy systems, including mechanical stability when motorized stages and autofocus drives are required, image analysis problems caused by non-uniform illumination and cell-to-cell unmixing, sample and liquid handling, 25 and above all, low throughput and yield in cell sorting.On the other hand, the development in mobile devices, especially in cell-phone cameras, has facilitated several cost-effective and field-portable imaging technologies, including lens-free optofluidic microscopy and cell-phone-based optofluidic fluorescent imaging cytometry. 26,27These miniaturized and automated lab-on-a-chip platforms provide commendable image quality and even three-dimensional cell representations in some examples. 28,29Yet, these systems are most suitable for a relatively small sample volume and cannot match the high throughput of conventional flow cytometers.
A parallel microfluidic cytometer (PMC) has been implemented to compromise between high throughput and high content, but has very limited 1-dimensional spatial resolution to resolve many sub-cellular components and structures compared to 2-dimensional cell images. 30,31Recent advances in imaging technologies, electronics, and digital computing have enabled imaging flow cytometry (IFC). 32,33s an integration of fluorescence microscopy and conventional flow cytometry, IFC combines flow cytometry's singlecell identification and high throughput with microscopy's cell image acquisition.Therefore, it becomes an ideal approach to simultaneously fulfill both analysis of morphological characteristics and phenotypic characterization of single cells within an enormous and heterogeneous population. 34][37][38] The basic idea behind IFC is scaling up flow cytometric spatial resolution to analyze more properties of cells.IFC, in this review, aims at the fluidic-based platforms that have optical imaging functionality at informative spatial resolution while retaining the main features of conventional flow cytometry.This article aims at discussing recent IFC technologies and also those advances developed targeting at IFC.The following sections will describe recent IFC technologies, review their strengths and challenges, and also discuss the outlook for IFC.

Technologies and methods
Taking both high throughput, i.e. high temporal resolution, and high spatial resolution into account, signal detection is the key challenge due to the fundamental trade-off between acquisition speed, sensitivity, and amount of information.Detectors used in imaging platforms can be divided into two types: 1) multipixelated imaging devices, such as chargecoupled device (CCD) and complementary metal-oxidesemiconductor (CMOS), and 2) single pixel photodetectors, such as photomultiplier tube (PMT) and avalanche photodiode (APD).The rest of this section will discuss IFC platforms that employ these two types of detectors separately.

Camera-based IFC
CCDs and CMOSs have a dense array of individual sensors in a 2-dimensional (2D) arrangement.Therefore, in an IFC system that uses CCD or CMOS as a detector, by employing wide-field illumination, 2D cell images can be produced as long as a sufficient number of photons are sensed within a given exposure time by those individual sensors of such camera devices placed at the image plane.The conundrum in this case is to increase the speed of such imaging tools.When applied in IFC that is on a cell-to-cell basis, the field of view is typically only 10s μm by 10s μm, so the large number of pixels in a CCD or CMOS camera is wisely re-arranged to work in parallel, either spatially or temporally.
2.1.1.ImageStream.The IFC developed by Millipore, e.g.ImageStream and FlowSight, relies on high-speed CCD cameras that use the time delay and integration (TDI) technique, which is originally designed to image objects moving along one axis at low light levels. 32,39,40This type of CCD provides higher sensitivity by having multiple rows of sensors which shift their partial measurements to the adjacent row synchronously with the motion of the moving cell image across the array of sensors.Applying this reading out technique, one can detect weak fluorescence signals without motion blur caused by an increase in exposure time.The sensor arrays on CCD are divided into N columns to detect emission or scattered light of N different spectral ranges from cells.Using spectral decomposition elements, 12 images per cell can be acquired simultaneously.Fig. 1 illustrates how the optics of ImageStream/FlowSight works.Because of the rich subcellular information acquired by ImageStream, various analysis and machine learning algorithms can be applied to study cell phenotype and subgroup classification.
Because the translation of the cell is exactly synchronized with the vertical charge transfer of each pixel on the CCD, using the TDI reading out technique requires a closely controlled fluidic system to ensure cells are centered and flow at a constant speed without rotation.This strict requirement hinders the system to adopt a sorting mechanism, since any minor fluidic disturbance from downstream cell sorting can cause imaging instability.One limitation of the system speed is the inherent data downloading method of CCD: every unit sensor passively collects incoming photons and stores electrons till the whole line/array has been read out; accumulated charges are transferred from the unit sensor to its neighbor before they are dumped into the charge amplifier to be converted into voltage.This working scenario constrains the system's data access rate.On the other side, obtaining enough sensitivity without any gain like electron multiplication also prevents the system to reach throughput higher than 3000 cells per second.
2.1.2.Multiple field of view.Instead of faster sophisticated photodetectors, a method named multi-field-of-view imaging flow cytometry (MIFC) was developed to image multiple changes simultaneously to obtain high throughput.This method circumvents the trade-off between throughput and exposure time by projecting multiple fields of view onto the CMOS camera. 41The increase of throughput is, therefore, proportional to the degree of parallelization, i.e. the number of parallel fluidic channels or the number of isolated fields of view, while the cell flow velocity is kept at a moderate level.Due to microfabrication of multiple parallel microfluidic channels and on-top N × M microlens arrays, the total field of view has to be very wide to cover N × M channels.Meanwhile, a diffractive lens made of polydimethylsiloxane (PDMS) is used for its good monochromatic performance.Fig. 2 shows the optical setup and the design of the lens array.
Despite that having more parallel fields of view means higher throughput, it is not always extendable due to field ab-erration caused by the diffractive lens.While field aberration is relatively less severe in MIFC systems since cells can be hydrodynamically focused and the field of view does not need to be significantly larger than single cells, typically 20 μm, chromatic aberration brought about by the diffractive lens makes MIFC work only in monochromatic mode.Light of various wavelengths has various light paths in an optical system that contains a diffractive lens.Its incompatibility with multi-mode and multi-spectral imaging becomes a hurdle of this IFC technique.
2.1.3.Temporally coded excitation.The primary way to image weak cell fluorescence signals in a CMOS-based system is to increase the exposure time, but consequential motion blur can dramatically downgrade the image quality.Motion blur occurs when the exposure time is longer than the time it takes the flowing cell to move a minimum resolvable distance.A technique that exploits temporally coded excitation effectively eliminates motion blur for fluorescence imaging of flowing cells. 42Instead of continuous illumination, a chopper wheel is used to generate modulated excitation pulse sequences with a pseudo-random code (shown in Fig. 3).In this way, the captured images can be processed with a known code sequence and known point spread function and finally be de-blurred using computational algorithms.Based on this technique, a fluorescence image of cells moving faster than motion-blur velocity can be reconstructed.In addition, since the decoded image and moving velocity of the object are found computationally, the system can compensate for the effects of velocity variation in a laminar flow.

PMT-based IFC
PMTs have superb sensitivity at photon counting levels and high dynamic range because of their internal tandem electron multiplication and gain adjustment.PMTs can also provide higher bandwidth and lower dark noise than CCD/ CMOS cameras to support high-throughput IFC systems.However, the readout of single-pixel PMTs presents the number of photons detected only in the time domain, which contains no spatial information.In some high-speed microscopy  laser scanning cytometry techniques, PMTs are combined with laser spot scanning to collect the entire light emitted or scattered by the illuminated cell and output a temporal signal.Cell images are then retrieved by assembling the intensity signal in the time domain according to the laser scanning position.Therefore, the overall throughput is limited by the speed of serial beam scanning.Instead of relying on mechanical beam scanning, several techniques have been developed to transfer spatial information to either the frequency domain or time domain in order to make use of PMT's extra bandwidth that has not been fully utilized in conventional flow cytometers.
2.2.1.STEAM.4][45] Different from the light sources, including lightemitting diodes (LED), laser, and mercury lamps, used in conventional flow cytometry or fluorescence microscopy, a mode-locked femtosecond pulse fiber laser is used to generate illuminating near-infrared light with a wide spectral bandwidth of 10s nm centered at ∼1000 nm wavelength (Fig. 4).The ultrafast broadband laser pulses are spectrally encoded by using an optical spatial disperser so that a 1D or 2D spectral rainbow for illumination is generated.In this way, the one-to-one spatial-to-spectral relation is obtained.In other words, every individual point of the cell is illuminated by light at a specific wavelength.After being recombined by the spatial disperser and temporally stretched by a dispersive medium, typically a long optical fiber, the transmitted rainbow is detected by a single-pixel photodetector.The output temporal waveform, therefore, represents the spatial information encoded by the cells being illuminated.IFC based on the time-stretching method can achieve throughput as high as 100 000 cells per second.In addition to oneto-one spatial-spectral mapping, encoding one location with multiple wavelengths is also available for more efficient measurements. 46esides producing bright-field images, STEAM can produce phase-contrast images of cells by generating two 1D orthogonally polarized spectral rainbows for illumination. 47,48owever, due to the high attenuation in the visible spectral range of one key optical component applied in the timestretching-based system-optical fiber-and the incoherence of fluorescence, this technology has not been able to produce fluorescence images, which is an essential function in numerous applications of cell analysis.Incorporating 1D fluorescence detection by employing additional lasers enables the system to retain features of conventional flow cytometry, but such fluorescence signals do not benefit from the timestretching technique. 49,50Another bottleneck of this technique, also due to the limited working spectral range up to the near-infrared regime (i.e.around 1000 nm), is that the spatial resolution does not go beyond the wavelength-related diffraction limit.
2.2.2.FIRE.Adopting the schemes in radiofrequency communications, an emerging technique named fluorescence imaging using radiofrequency-tagged emission (FIRE) has made high-speed fluorescence imaging possible. 51,52he continuous-wave laser is converted into multiple intensitymodulated excitation beams by using an acousto-optic deflector (AOD) and an acousto-optic frequency shifter (AOFS) in the optical setup (Fig. 5).Every individual point of the cell within the imaging field of view is excited by light modulated at a distinct radio frequency.Acousto-optic components diffract light to different angles in 1D, and cells move at a certain speed along the direction orthogonal to the acoustooptically scanned beam in an IFC system.Thus, each pixel in a 2D cell image captured by FIRE is represented by a unique combination of one radiofrequency and one time-point.The resulting throughput is up to 50 000 cells per second.In addition, by increasing the bandwidth from that of the employed AOD, the speed of this imaging technique is further exploited to be limited by fluorescence lifetime. 53ecause the time-domain signal from a FIRE system is a Fourier superposition of the radiofrequency-tagged emission from one row of pixels, a Fourier transform is required in data processing.This can make realization of real-time processing and instant result generation difficult for the system.Besides, the operations of the key components in a FIRE system-AOD and AOFS-are wavelength-dependent, which makes multicolor imaging very challenging.In addition, because this radiofrequency tagging technique is only applied to coherent light sources, the FIRE system is not suitable for bright-field imaging where an LED source is preferred to avoid speckle noise and interference.
2.2.3.Spatial-temporal transformation.Without sophisticated optical components for fluorescence excitation and detection or multipixelated photodetectors, a technique named spatial-temporal transformation is applied to retrofit a conventional flow cytometer into an IFC system with minimal modification. 54,55The central, defining feature of this technique is its ability encode the time-domain signal waveform with a specially designed spatial filter so that the waveform consists of a sequence of patterns separated in the time domain.By inserting the spatial filter with a known pattern in the image plane, fluorescence, transmission and scattered light from different parts of the cell pass through different slits at different times (Fig. 6).Cell images in multimodes, such as fluorescence and scattering, can be assembled by using algorithms corresponding to the spatial filter used.However, the throughput of the design is inversely proportional to the length of the spatial filter, and the spatial filter cannot be arbitrarily shortened in order to attain decent spatial resolution.Without losing any features of conventional flow cytometry, this technique enables the incorporation of multi-mode and multi-spectral imaging capabilities.Moreover, the optical configuration of this IFC system allows the use of disposable microfluidic devices; [56][57][58] the involved computation requires minimal time, and unlike the CCD-based IFC, the data access time from the PMT does not limit the system's overall imaging and processing throughput.Therefore, this technique is well suited for high-throughput, realtime image-based cell classification and sorting.
In addition, by adding more PMTs, dichroic mirrors and more light sources, the system has been extended to work in a multi-parameter manner-multicolor fluorescence, brightfield and dark-field imaging.As shown in Fig. 7, two-color fluorescence, transmission and backscattering cell images are demonstrated.

Challenges: high-throughput and real-time image analysis
0][61] There are many software packages and tools for use in high-throughput image analysis, mainly for microscopy platforms, including CellProfiler and ImageJ, but in general, the pipelines of these tools are for offline image analysis. 62Fortunately, some of  difficult image processing problems in high-throughput microscopy and LSC can be avoided in IFC.For example, due to cell-to-cell and cell-to-substrate adhesion, strategies for image segmentation such as threshold, watershed and edgedetection have been under development for decades, but still a bottleneck of the automated image analysis in microscopy.4][65][66][67] For blood cells, bone marrow cells, and many cancer cells flowing in the blood vessels, IFC is the most promising approach to study their morphological changes.
Compared to the data format in conventional flow cytometry, including integral, peak and width of light intensity, cell images produced by IFC are much more complex.Since IFC can produce thousands of multi-spectral cell images per second, files generated by IFC can tremendously burden the dig-ital image transportation and processing realized by the back-end data handling unit.Assuming that a field-of-view of 40 μm by 40 μm is represented by an image of 100 pixels by 100 pixels, at least 100 MB data are generated in one second at a throughput of 10 000 cells per second.Therefore, a test of a few minutes can easily create a data file beyond 10s of GB.For a possible solution, a compressive sensing theorybased method has recently been explored to build analog compression directly into the acquisition process so that the sampling can be significantly more efficient. 68,69Some machine learning techniques can also be applied to data processing for richer information carried out by IFC systems. 66,70omputational requirements for IFC platforms are unprecedented.All IFC systems demonstrated to date perform image analysis offline.In order to combine cell sorting with IFC to fully realize its tremendous potential, real-time image  construction and analysis is Hence, the ability to produce, measure, and analyze cell images, and sort cells in a real-time manner will be the next major milestone for IFC.
Possible approaches to extract cell characteristics in real time include the use of field-programmable gate arrays (FPGA) or graphics processing units (GPU) to implement various image processing and machine learning algorithms.

Conclusion
With the advent of big-data era, life scientists started to grapple with massive volumes of data.Recent advances in highthroughput IFC allow continuous high-throughput capture of cell images.Various technologies have made the generation of multiparametric imaging files highly feasible although efficient analysis and utilization of this huge amount of data remain a challenge.Multimodality (i.e.transmission, scattering and fluorescence), functional flexibility (i.e.operation as conventional flow cytometry or IFC at users' choice), and compatibility with cell sorting are the three principal areas of development for IFC to gain wide acceptance as a workhorse for biomedical research and clinical applications.

Fig. 1
Fig. 1 Optics of ImageStream.Reproduced from ref. 40 with permission from the EMD Millipore Corporation.

Fig. 2
Fig. 2 Multiple field-of-view imaging flow cytometer.(a) Diffractive lens wide-field imaging system.L1 collimates the LED.L2 is a condenser that images Iris 1 onto the object plane.DL is the diffractive lens providing multiple fields of view.L3 is a relay lens.(b) 16 object planes.(c) 16 imaging planes.Sample: 3.5 μm latex beads.Reproduced from ref. 41 with permission from the Royal Society of Chemistry.

Fig. 3
Fig. 3 Coded excitation fluorescence microscope.(a) Chopper wheel that modulates the excitation beam with a pseudo-random code.(b) A microfluidic device imaged using a 40× fluorescence imaging microscope.(c) Raw blur encoded images captured by the camera and decoded images after the computational approach.Reproduced from ref. 42 with permission from the Optical Society.

Fig. 4
Fig. 4 Schematic of the STEAM flow analyzer.Imaging and illumination optics takes blur-free images by encoding object location information into the spectral domain.ADFT converts spectral information into the time domain through the time-stretching method and processed using a real-time imaging processor.Reproduced from ref. 45 with permission from PNAS.

Fig. 5
Fig. 5 FIRE microscopy.(a) Schematic diagram of the FIRE microscope.BS, beam splitter; AOD, acousto-optic deflector; AOFS, acousto-optic frequency shifter; DM, dichroic mirror; EF, fluorescence emission filter; OL, objective lens; PMT, photomultiplier tube; DIG, 250 MS digital recording oscilloscope; RS, resonant scanning mirror.Upper inset: the AOD produces a single diffracted first-order beam for each radiofrequency comb frequency.Lower inset: beat frequency generation at the MZI output.(b) Gabor lattice diagram of FIRE's frequency-domain multiplexing approach.Points in the same horizontal line are excited in parallel at distinct radiofrequencies.The horizontal line is scanned by a galvo-mirror to acquire a 2D image.Reproduced from ref. 51 with permission from the Nature Publishing Group.

Fig. 6
Fig. 6 Implementation of spatial-temporal transformation-based IFC.(a) Schematic diagram of the imaging flow cytometer system.DM, dichroic mirror; SF, spatial filter; EF, emission filter; PMT, photomultiplier tube.(b) Spatial filter design that has ten 100 μm by 1 mm slits positioned apart in a way that one is immediately after another in both the x-direction and y-direction.(c) Experimental result: time-domain PMT output signal of fluorescent light from an A549 cell stained with CellTrace CFSE, corresponding original fluorescence image restored using an algorithm, and corresponding resized fluorescence image to show the real size of the cell.The numbered regions segmented by dashed lines demonstrate the correspondence between the timedomain signal and the resulting image.Size is labelled in the figure.Reproduced from ref. 54 with permission from the Nature Publishing Group.

Fig. 7
Fig. 7 Demonstration of bright-field, two-color fluorescence and backscattering cell images.Scale bars are 5 μm.(a) Bright-field images of MDA-MB-231 human breast cancer cells flowing at 0.2 m s −1 .(b) Representative two-color fluorescence images of MDA-MB-231 human breast cancer cells stained with CellTrace CFSE, with 1 μm fluorescent beads attached to the cell membrane, flowing in the microfluidic channel at 0.25 m s −1 . 47(c) Representative fluorescence plus backscattering cell images from spatial filter-based imaging flow cytometry.All images are of A549 human lung adenocarcinoma epithelial cells, stained with CellTrace CFSE, flowing at a velocity of 0.2 m s −1 .Reproduced from ref. 54 with permission from the Nature Publishing Group.

Table 1
Summary of imaging flow cytometry (IFC) techniques FC: conventional flow cytometry.