Recent approaches for optical smartphone sensing in resource-limited settings: a brief review

Developments in the emerging ﬁ elds of smartphone chemical and biosensing have dovetailed with increased interest in environmental and health monitoring for resource-limited environments, culminating in research toward ﬁ eld-ready smartphone sensors. Optical sensors have been a particular focus, in which smartphone imaging and on-board analysis have been integrated into both existing and novel colorimetric, ﬂ uorescent, chemiluminescent, spectroscopy-based, and scattering-based assays. Research in recent years has shown promising progress, but substantial limitations still exist due to environmental lighting interference, reliance upon proprietary smartphone attachments


Introduction
Over the past decade, the revolution and widespread adoption of smartphone technology has spurred research interest in designing smartphone-integrated point-of-care (POC) chemical and biosensors, the history and development of which are extensively surveyed in several topical reviews. [1][2][3][4][5][6] On their own, these portable sensors are designed to identify the presence and concentration of chemical and biological targets, or analytes, based on their interactions with target-specic agents, such as dyes, enzymes, antibodies, and aptamers. These interactions are expressed and thus sensed in a number of ways, including optically, as with colorimetric or uorescent chemo-and immunosensors, or electrically, as with potentiostat-controlled or interdigitated microelectrode immunosensors. [7][8][9][10][11] Smartphone devices have been used in concert with both platform varieties, serving in roles between simple readers and instrument interfaces to fully-integrated sensors, acting as detectors, data processors, and even signal inducers in one pocket-sized package. 2,3,6,[12][13][14] With rapid improvements in smartphone optical hardware, though, research has favored the exploration of smartphones as standalone optical sensing platforms for colorimetric, uorescent, spectroscopic, scattering, and microscopy-based applications. 6 Overall improvements in smartphone on-board processing, dual-camera designs, as well as GPS and mobile data connectivity have evolved in parallel with increasingly low-cost devices for emerging markets worldwide. [15][16][17] Thus, recent research has expressed particular interest in adapting POC smartphone sensing techniques for "resource-limited" settings, such as eld sites in rural or developing regions. 5,[18][19][20] These circumstances are removed from traditional laboratory infrastructure, such as electricity, refrigeration, sanitation, controlled lighting, and stationary precision equipment. 21 However, outside of a controlled environment context, variable factors such as ambient lighting, temperature, and humidity can interfere dramatically with chemical and biosensor assays. [22][23][24][25][26] Additionally, in-eld or in situ sample matrices are frequently complex in nature, containing solid or dissolved materials such as dust, algae, or proteins, that can potentially interfere with or shroud analyte-reagent signals. Much research has led to situational-specic optimized platform layouts, sample preparation techniques, and normalization procedures that help to build on the strengths of smartphones as sensors, but limitations remain in their adaptability. Despite improvements since their early mainstream adoption, smartphones still lack the full sensitivity of designated sensor instrumentation. Compensation must be made for these limitations, and this begins with an assessment of the overall challenges that recent sensing methods have faced.
In this review, we hope to illuminate how recent research in smartphone chemical and biosensing has strived to accommodate robust eld detection under disruptive conditions, and discuss the potential technical challenges that must still be overcome to allow sensing in resource-limited environments. Although great strides have been made within individual research groups and among individual sensor targets, a more concerted and broader approach is needed if we are to reach our ultimate goal of truly universal smartphone sensors for pocketsized and all-in-one data collection, processing, and rapid results-sharing.

Smartphone-integrated sensor platforms
Smartphones are typically integrated into an optical chemical or biosensing platform through the use of an attached lighting control enclosure that closely ts the device and the assay platform (i.e. strip, cassette, cuvette, etc.) (Fig. 1). In this conguration, the smartphone can function as an illumination source (LED ash), a signal detector (CMOS-based camera), and a signal processor (soware application or app). Based on the conditions of the assay, various lenses, lters, diffraction gratings, and alternative light or power sources may be incorporated within the attached enclosure to enhance signal detection from colorimetric, uorescence-based, chemi/bioluminescencebased, and scattering-based assays. There are, however, several exceptions that utilize image-processing algorithms rather than enclosure attachments. These are further discussed in the Challenges and future needs section. [27][28][29] Comparison of these four optical sensing modalities is graphically illustrated in Fig. 2.

Colorimetric sensors
In their basic form, colorimetric assays evaluate changes in the absorbance or reected intensity of analyte-reagent complexes. These changes are commonly due to structural shis or plasmon resonance phenomena which lead to a shi in the sample's optical properties typically over a wide range of wavelengths. 30,31 Since conventional smartphones rely on a CMOS array to assign RGB values to each pixel that it registers, the smartphone is perhaps best equipped to monitor for these broad color shis since they do not require stringent controls or lters, which can be needed in uorescence detection to screen for ne changes at a peak wavelength. Thus, simple  colorimetric smartphone sensors only require lighting and image processing for unltered detection (Fig. 1a). More complex colorimetric smartphone sensors may also incorporate additional lenses and separate lighting sources more specically attuned to the analyte-reagent absorbance spectrum (Fig. 1b).
Liquid matrices potentially benet lower assay limits of detection due to the increased path length and less restricted diffusion that is afforded for light through a homogenous solution as opposed to reection from a solid surface sample. According to the Beer-Lambert law, absorbance is directly proportional to path length. Thus, for absorbance changes in colorimetric assays, light transmitted and detected across a liquid sample, with a path length on the order of millimeters or even centimeters, can inherently produce a greater signal response than the same signal detected through a liquid lm, which may only be microns across. This is provided that the absorbance or scattering of other components in the sample does not reduce the transmissivity of the whole solution, as is problematic for whole blood matrices. 28,51 These analyte-reagent colorimetric signal changes require some time for stabilization, however, with assays using larger sample volumes and/or immobilized reagents requiring 5-15 minutes or up to an hour for development (Table 1). In this manner, mobile-phase strips or thin hydrogel lm assays provide shorter assay times, as they reduce diffusion distances while still permitting individual mobility of both the analyte and reagent, overcoming the main source of temporal limitations at the cost of path length. An ideal smartphone colorimetric assay platform might nd the optimal union of these two, allowing detection across a longer path length but with a low total volume to reduce diffusional distances when considering low concentrations of analytes.

Fluorescence-based sensors
Smartphones are also popularly integrated with uorescencebased assays (Table 1). These detect light emitted upon the radiative excitation and decay of a target-uorophore complex. The photons released within this signal response are distributed across a relatively narrow band of peak emission wavelengths, which can be amplied to allow potentially lower thresholds of target concentration detection. 58,59 Additionally, as uorescence is directly proportional to the intensity of the excitation source, the spectrum and intensity of sensor illumination can be tuned to further reduce the limit of detection as compared with colorimetric assays. 26,60 These strong and narrow emission peaks can also potentially benet multiplexing when targets are uorescently tagged separately within a single sample. [61][62][63] However, because a smartphone CMOS array is unable to fully distinguish between neighboring wavelength responses, long pass lters are typically inserted between the assay platform and the smartphone lens to allow only emitted light to reach the CMOS (Fig. 1c).
Additional research has designed novel, high-performance lters and materials that can be paired with mobile assays to improve light rejection both spectrally, ensuring that light outside of the target emission wavelength does not interfere with detection, and also spatially, controlling the scattering and diffraction of emitted light. These can be vital for low concentration detection and for sensing applications that are spatially sensitive, as with smartphone uorescence microscopy. Examples of these advanced lters and surfaces include the "silo-lter" developed by Lee et al., which operates using a grid of light absorbent cells that ts over a CMOS array to improve the spatial resolution for sensing discrete cells. 64 Another example is the multilayered photonic crystal surface developed by Ricciardi et al. 65 Through controlled diffraction, this offers the advantage of 40 times lower detection limits over detection from comparable surfaces and allows the use of collectors with a lower numerical aperture, which can be potentially less expensive and reduce residual aberrations in the signal. As another option, Wargocki et al. chose to use a polarizer to separate excitation and emission light between a tablet computer screen, which served as an excitation source through rear-illumination of wells containing collagenase-and trypsin-uorescein, and a smartphone camera detector. 66 This allowed low limits of detection (3.75 mg mL À1 collagenase, and 3.72 ng mL À1 trypsin) despite intense proximal excitation light from the tablet computer.
Recently, uorescence-based detection has been widely used for biological targets, such as bacterial and viral antigens, proteins, nucleic acids, and mycotoxins, 62,66-75 but have also been used for some chemical targets. 29,[76][77][78][79] In polymerase chain reaction (PCR) and loop-mediated isothermal amplication (LAMP), the uorescent emission of an intercalating dye allows identication of specic nucleic acid segments that have been amplied from an organism. [80][81][82][83] Fluorescent dyes that have been used to tag proteins and other cellular surface antigens have also been sensed through smartphone microscopy. 84, 85 Wei et al. demonstrated such uorescent detection paired with smartphone microscopy, which was used to allow counting of individual virus capsids. 69 As with colorimetric assays, uorescent smartphone detection has primarily been used for samples in liquid matrices using larger volume well plates or mobile state lm, cartridge, or slide platforms. Recently demonstrated assays have involved incubation and signal development times between one and ve minutes. 29,72,79 These have allowed for better real-time results, particularly when using an excitation light source for more immediate and on-demand uorescent detection as opposed to the time-dependent signal development and decay that occurs with some chemiluminescent reactions.

Luminescence-based sensors
Smartphones have also been used for these other luminescencebased assays (i.e. bioluminescence and chemiluminescence assays), which detect narrow-band light emission from biological or chemical reactions arising between a target and a reagent. While it is possible to use an excitation light source to enhance these assays, in principle it is not required due to the spontaneous light emission which fundamentally distinguishes these from uorescent and colorimetric assays. Such characteristics allow these assays to be highly sensitive to a given target. However, the intensity of a luminescent reaction is timedependent, decaying exponentially such that the intensity for some reactions is negligible aer only a few seconds. 78 This can lead to potential challenges in consistent quantication, but can be paired well with applications such as cell counting, cytotoxicity screening, or particle size discrimination, in which exact signal quantication may not be necessary. For example, Zhu et al. have paired luminescent smartphone-based sensing with ow cytometry to facilitate cell counting and particle size discrimination. 86

Spectroscopy-based sensors
Spectroscopy attachments have oen been paired with smartphones to work in coordination with colorimetric, uorescent, and other luminescent sensing methods because they allow a smartphone to distinguish wavelengths by spatially separating them across CMOS array. 87,88 This is typically accomplished through a grating that diffracts light aer it has been reected by or transmitted through the sample, but before it has entered the camera aperture (Fig. 1d). Hossain et al. used such techniques with dual absorbance and uorescence assays for pH and Zn 2+ , and it allowed them to separately analyze signal wavelengths at a resolution of 0.42 nm per pixel over a 300 nm bandwidth. 89 Recent research has also evaluated assays for proteins and antigens, 88,90 enzyme activity, 91 nucleic acids, 87 neurotoxins, 92 and pH 93 in similar manners.

Scattering-based sensors
Additional applications of smartphone optical sensors have focused on detecting elastic light scatteringthat is, particlebased scattering which results in a change in light direction, but no change in its energy (wavelength). 94 This form of scattering can be subdivided into the Rayleigh and Mie scattering regimes, which both formally describe the scattering of spherical particles. 95 The Rayleigh regime describes the multidirectional scattering of particles that are signicantly smaller than the wavelength of incident light (the size parameter a ¼ pd/l is (1, where d is the diameter of the scattering particle and l is the wavelength of the incident light). The Mie scattering regime more generally applies to all spherical particles, regardless of size. As such, it encompasses Rayleigh scattering; however, the mathematical complexity of Mie theory prompts the use of simplied Rayleigh scattering theory where applicable. For incident light at visible wavelengths, Mie theory may be used to generally describe the forward angular scattering characteristics of micron and sub-micron scale particles (d $ l). Both forms of scattering have recently been applied in smartphone sensing platforms, with perhaps more emphasis given to Mie scattering. The intensity of Mie scattering drastically changes, or "oscillates," over scattering angles and particle sizes. This can thus be used to detect particles based on size characteristics and to detect aggregation/agglutination within samples. Mie scattering has been recently applied for quantifying discrete particles in turbidity, 96 as well as for quantifying bacteria concentrations in water, meat, and urine samples. [97][98][99] While Rayleigh scattering also varies over the scattering angle, these changes are rather monotonic and may provide weaker signals for distinguishing particle size in aggregation/ agglutination-induced biosensing. However, Rayleigh scattering can provide strong sensitivity when used in conjunction with nanotechnology. Recent applications include laserinduced detection of lead by gold nanoparticles, 100 as well as a condensation-based nanolens which allows detection of nanospheres and nanorods of diameters below 40 nm and 20 nm, respectively. 101 Further work with scattering-based detection has evaluated label-free immunoassays of hepatitis B and HIV antigens. 102 Nicolini et al. demonstrated multimodal smartphone detection of bacteria via particle immunoagglutination, combining a scattering-based assay to identify the concentration of E. coli K12 and Salmonella Typhimurium, and a uorescence-based assay to allow for specicity between the two targets. 61

Smartphone microscopy
The nal application of smartphone sensing discussed here is smartphone microscopy. Some recent innovations have included work toward computational lensfree microscopy, while others have combined illumination and focusing attachments together with computational methods including convolution, super-resolution algorithms, and machine learning to allow for improved spatial resolution that would not otherwise be possible with the intrinsic optical components of the smartphone camera alone. An example of these combined techniques is CellScope, a smartphone microscope used for applications including ocular imaging and tuberculosis diagnostics, and which has recently been integrated with computational refocusing and illumination techniques using a domed LED array attachment. [103][104][105] This allows the user to conduct brighteld, darkeld, and phase contrast microscopy in real-time with axial resolution of $5 mm. Other steps in smartphone light microscopy have demonstrated recognition of targets including micron-scale Bacillus anthracis Sterne spores from growth media and helminth ova directly from stool samples. 106,107 Smartphone microscopy has also been woven together with other sensing modalities, especially with uorescence which can be used as a beacon for discrete targets. Koydemir et al. utilized smartphone-based uorescence microscopy for detecting Giardia lamblia cysts over a wide eld of view ($0.8 cm 2 ) at concentrations as low as $12 cysts per 10 mL directly from a water lter cassette. 84 Smartphone uorescence microscopy has also been used recently for rapidly distinguishing and counting red and white blood cells in ow cytometry applications, 86 and has allowed for detection of even smaller particles, such as virions and nanoparticles on the order of 10 nm despite the dimension constraints of the CMOS array itself. 69 Further in-depth discussion of smartphone microscopy can be found in other topic-specic reviews. [108][109][110][111] Challenges and future needs Overall, perhaps the most basic challenge in smartphone optical sensing is interference from uncontrolled or uneven lighting, which can fundamentally detract from the raw signal being detected. Common control measures include the use of enclosed lighting and imaging attachments for the smartphone, as well as collimating lenses or optical bers to reduce light dispersion. Similar controls, especially attached lenses, can help to improve sensitivity for smartphone microscopy. 69,106,112 Through their pairing with a grating, as in spectroscopy, the smartphone can also achieve sub-nanometer wavelength resolution across a sub-micron bandwidth in the smartphone viewing eld, as demonstrated by Yu et al. and Hossain et al. 87,89 Tied into these concerns are the physical constraints of the Bayer color lter array (CFA) in a typical smartphone CMOS image sensor. The Bayer CFA functions by providing three lters for incident light (red, green, and blue), which alternate along two orthogonal axes to allow intermingled transmittance of red, green, and blue light to a CMOS sensor to create the spatial perception of color within an image. 113 Because the light is separated into discrete units (pixels), the sensitivity of a smartphone optical sensor is ultimately limited by the number and sensitivity of these units. Again, lenses are frequently used to concentrate the signal emitted from a sensor platform before it reaches the smartphone optical hardware, and trade-offs in other parameters such as speed or imaging area and depth can be counterbalanced with improved sensor sensitivity.
Subhash et al. demonstrated a smartphone-based optical coherence tomography system that takes advantage of both a repurposed miniature optical delay from a compact disc optical pick-up head as well as a partial mirror to provide depthresolved imaging with resolution comparable to an equivalent commercial grade system for under $10, but at the cost of lower speed. 114 Balsam et al. paired both physical and computational enhancement for smartphone sensing of uorescent signals by way of a capillary tube array and image stacking/background normalization processes, reporting a 1000Â decrease in their observed uorescent limit of detection for an adenovirus DNA target. 115 While these various physical lighting control mechanisms provide the advantage of controlling uncertain surrounding conditions, though, they can also limit the particular sensing platform to a single device, which in turn limits the broad usability of the sensor platform. For resource-limited sensing, research that focuses on platform-independent or directly platform-translatable techniques, such as computational methods, can be benecial since reliance upon secondary attachments introduces additional costs, special material requirements, and platform limitations. Separate smartphone models may be available in different countries, and newer models with substantially improved cameras and soware technology are frequently introduced.
Work by Sumriddetchkajorn et al., Devadhasan et al., and Xu et al. are notable examples of this alternate focus, as they have each explored the use of image-processing algorithms and simple reference points within the imaging background to provide controls similar to those offered by attachments. [27][28][29] In these works, researchers have successfully demonstrated sensitive colorimetric quantication of dissolved O 2 /pH, pH, and chlorine, respectively, without the need for an attached enclosure. However, on the whole most current smartphoneintegrated sensors still rely on such enclosures for smartphoneintegrated colorimetric, uorescence-based, and scatteringbased assays.
It should also be noted that typical smartphone camera apps do adjust both its hardware and the acquired images to allow the images to look natural to human perception. This is done through "auto-focus", "auto-exposure", and "white balance" corrections, among other settings. While it is relatively easy to manually turn off the auto-focus and auto-exposure features, though, it is comparatively difficult to turn off or adjust the white balance feature. White balance corrections are specically problematic in colorimetric and uorescent sensing, since the RGB pixels in a CMOS array are not entirely perfect in recognizing red, green, and blue colors. For example, red pixels can still exhibit a limited response to blue colors, thus requiring corrective measures. In addition, human eyes are imperfect in their recognition of the three basic colors. Both red and blue cone cells in the human eye respond to $420 nm, generating a violet color which looks very similar to purple, a mixture of red and blue colors. When smartphone apps are designed to mimic human perception, they can also mimic these imperfections of the human eye. In addition, it is known that the "tones" of images vary between smartphone manufacturers. Therefore, a complex normalization algorithm must be implemented and/ or an app that saves the raw images as they are acquired should be used.
Spatial lighting gradients and variance can also be a serious concern in optical smartphone sensing. Regardless of the light sources (ambient lighting or a separate light source), the level of lighting can vary over a single image. In addition, the four corners of a typical smartphone image tend to be darker than the other areas. In this regard, complex normalization algorithms have been adapted to overcome lighting bias errors and extract select signals or features of interest even in unltered and broad-eld images. 116 A fast Fourier transform (FFT) lter and triple normalization algorithm were successfully utilized to minimize the ambient lighting bias, spatial imaging bias, and inter-and intra-channel variations from multi-channel micro-uidic paper analytic device (mPAD) for smartphone-based water quality monitoring (Fig. 3). 117 These are limited by the sensitivity of the smartphone optical instrumentation itself, however. A better approach for severe lighting bias would be to provide a normalization algorithm that can function under a more universal lighting control region, such as a known reference background of simple unattached box enclosuresomething that might be accessible to anyone anywhere. Recent research has explored these possibilities, but more work is needed to test such methods under rigorous lighting conditions. [27][28][29] Similarly, more research is needed to evaluate differences between individual smartphone platforms. Although smartphone optical hardware is frequently sourced third-party from similar manufacturers, their congurations within the nal smartphone may still differ dramatically, affecting signal measurements. Doeven et al. compared the functional hardware differences between ve phones, albeit with a focus on audio outputs designed to mediate chemiluminescence. 77 This evaluation identied substantial differences between phone characteristics, even among a single manufacturer. Askim et al. offered a comparison between smartphone (iPhone 5s) sensing and three additional handheld sensing methodsa novel handheld CMOS-based CIS reader, a consumer-grade atbed scanner, and a consumer-grade DSLR camerafor colorimetric assay detection. 118 They found that the absence of lighting control mechanisms in this standard smartphone platform actually led to higher signal deviation as compared with all other methods, with the handheld and atbed scanners resulting in the greatest consistencypurportedly on account of uniform controlled lighting capabilities and a constant focal distance. In light of these conclusions, investigation into the consistencies between many separate smartphone devices would be ideal to identify how different smartphone light sources and cameras separately analyze chemical or biosensor assays. This would strongly inform the design of control measures for improved resource-limited sensing.

Conclusions
Across recent research, interests in a diverse range of eld applications have brought along unique techniques and devices in the hope of enhancing smartphone sensitivity for instrument-grade quality. The creativity in these advances has brought many promising results, but the broad applicability of this work in many resource-limited settings is still a challenge due to platform limitations, with oen uncertain reproducibility across smartphone devices and with potential limitations in the use of advanced smartphone attachments, which may not be easily or directly translatable between devices. Certainly such constraints are necessary for optical sensing techniques that require precise alignment of components, such as smartphone spectroscopy or microscopy, but interesting and practical insights may be offered by further exploration of reductionist designs and platform-translatable computational techniques. If feasible, such measures could lead toward broader future implementation serving a wider base of users in resourcelimited settings. Fig. 3 (a, left) Across a smartphone image, unfiltered lighting contributes to anisotropic illumination and shading that can lead to measurement errors between identical channels, as seen both horizontally and vertically across blank microfluidic paper analytical device channels. (a, right) Image processing techniques even as simple as a multi-reference normalization algorithm or a fast Fourier transform (FFT)-based can help to dramatically reduce such errors. Further work in this area can help to eliminate the need for secondary smartphone attachments to provide consistency both (b) between sensor measurements, and (c-e) along different regions of even a single sensor device.