Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Leveraging a smartphone to perform time-gated luminescence measurements

  • Brian E. Fratto ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Validation, Writing – original draft, Writing – review & editing

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Emma L. Culver ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Formal analysis, Investigation

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Gabriel Davis ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Investigation, Software, Validation

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Robert Deans ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Writing – original draft, Writing – review & editing

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • John B. Goods ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Investigation, Writing – original draft, Writing – review & editing

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Sean Hwang ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Formal analysis, Software

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Nicole K. Keller ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Investigation, Methodology, Validation, Writing – review & editing

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • John A. Lawrence III ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Investigation, Validation

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Alexander R. Petty ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Investigation, Methodology, Validation

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Timothy M. Swager,

    Roles Conceptualization, Writing – review & editing

    Affiliation Department of Chemistry, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America

  • Joseph J. Walish ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Conceptualization, Investigation, Validation, Writing – original draft, Writing – review & editing

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Zhengguo Zhu ,

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Data curation, Investigation, Methodology, Validation

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

  • Jason R. Cox

    Contributed equally to this work with: Brian E. Fratto, Emma L. Culver, Gabriel Davis, Robert Deans, John B. Goods, Sean Hwang, Nicole K. Keller, John A. Lawrence III, Alexander R. Petty, Joseph J. Walish, Zhengguo Zhu, Jason R. Cox

    Roles Conceptualization, Data curation, Writing – review & editing

    jason@c2sense.com

    Affiliation C2Sense, Inc., Watertown, Massachusetts, United States of America

Abstract

Empowered by advanced on-board sensors, high-performance optics packages and ever-increasing computational power, smartphones have democratized data generation, collection, and analysis. Building on this capacity, many platforms have been developed to enable its use as an optical sensing platform for colorimetric and fluorescence measurements. In this paper, we report the ability to enable a smartphone to perform laboratory quality time-resolved analysis of luminescent samples via the exploitation of the rolling shutter mechanism of the native CMOS imager. We achieve this by leveraging the smartphone’s standard image capture applications, commercially available image analysis software, and housing the device within a UV-LED containing case. These low-cost modifications enable us to demonstrate the smartphone’s analytical potential by performing tasks ranging from authentication and encryption to the interrogation of packaging, compounds, and physical phenomena. This approach underscores the power of repurposing existing technologies to extend the reach and inclusivity of scientific exploration, opening new avenues for data collection and analysis.

Introduction

The ubiquity of smartphones, combined with the fusion of advanced camera, computational and sensing hardware, has changed the lives of billions worldwide [111]. The ability to communicate freely, create and stream live videos, and synchronously store and retrieve data are now widespread thanks to these advances [1, 3, 4]. The applicability of smartphones for portable scientific use cases is only recently emerging as an area of active commercial interest [26, 810] despite the obvious advantages of combining portable communication capabilities with sensor fusion. For this vision to come to fruition, the smartphone’s pervasive convenience must merge with the robust and precise measurement functionalities found in laboratory-grade instrumentation [5, 12].

Fluorescence spectroscopy is a marquis laboratory analytical technique for the trace detection of chemical and biological substances [13]. Predicated on molecular excitation and the subsequent measurement of luminescence, fluorescence-based techniques exhibit remarkable selectivity and sensitivity. These properties have enabled its use in many diverse fields, including biochemistry, molecular biology, environmental monitoring, and diagnostic platforms [1417]. Yet, its complex and costly nature often restricts fluorescence spectroscopy largely to laboratory environments—the availability of portable, accessible variants remains elusive [1821]. Traditional fluorescence measurements often rely on the quantification of emission intensity signals [22]. This is easily accomplished in laboratory settings where the optical geometry and lighting conditions are well controlled and sample preparation is easily accomplished prior to measurement. In the field, however, these measurements become markedly more difficult due to the non-standardized nature of interrogation [23, 24]. Reliable quantification of emission intensity using a smartphone would require the adoption of unchanging and well-calibrated ratiometric signals to mitigate the impact of the inverse square dependence of optical intensity on distance [23] Compounding the measurement further is the existence of multiple sources of autofluorescence that are ubiquitous in the field.

Time-gated spectroscopy, however, offers a promising alternative [2535]. By measuring the emission lifetime of a luminescent material, this technique retains many of the benefits associated with fluorescence spectroscopy, namely exquisite sensitivity and selectivity, while proving less susceptible to radiative interference and eliminating the need for ratiometric signals [36, 37]. Since the luminescent substance’s lifetime is unique and typically invariant to read angle and/or ambient lighting conditions it offers resilience. Most importantly, measurements of delayed emission provide a means to overcome the negative effects of autofluorescence from the analyte’s local environment, one of the most significant challenges in realizing luminescence measurements outside of the laboratory [38]. By waiting for the relatively fast emissions of surrounding materials to decay before analysis, the signal to noise ratio of the measurement is significantly improved. Smartphone-based luminescent readers have been reported [3952], their inability to conduct time-resolved measurements leaves them prone to the parent technique’s limitations. Consequently, we set our sights on enabling common smartphones—via low-cost enhancements to their native hardware and software—to perform time-delayed measurements, thus capitalizing on these ubiquitous devices’ inherent features in a cost-effective, user-friendly manner.

The current generation of smartphones, armed with a Complementary Metal-Oxide Semiconductor (CMOS) imaging chip, offer exceptional imaging capabilities at a consumer-friendly price point [53]. Despite their mass deployment, CMOS-based cameras have their limitations, chiefly their lack of a memory buffer, which confines the exposure process to an electronic ‘rolling shutter’ mechanism [43, 54, 55]. This mechanism operates through the sequential exposure and readout of each photodiode array row, resulting in a delay from the first to the last pixel row. Put another way, the rolling shutter mechanism means each row of the photodiode array is sampling a different time domain than the adjacent rows before and after. This lag often manifests as banding and geometric distortion, particularly when capturing fast-moving objects or intermittently flashing LEDs [54, 56, 57],—generally considered a photographic disadvantage [58]. Recently, we [59, 60] and others [52] have shown that these limitations can be exploited as features to perform time-delayed fluorescence measurements. In this paper, we extend the applicability of this technique to sensing and authentication via the development of a fully integrated smartphone device.

Materials and methods

Chemicals

The luminescent materials used in this manuscript are: (1,10-phenanthroline)tris[4,4,4-trifluoro-1-(2-thienyl)-1,3-butanedionato]europium(III) (Synonyms: Eu(tta)3phen), terbium 2,2,6,6-tetramethyl-3,5-heptanedionate, poly(2,5-di(3’,7’-dimethyloctyl)phenylene-1,4-ethynylene), N,N’-bis(1-hexylheptyl)perylene-3,4,9,10-tetracarboxybisimide, poly(9,9-dioctylfluorene-alt-benzothiadiazole) (CAS: 210347-52-7), and 4,4′-bis(9-carbazolyl)benzophenone. Where applicable, polymer matrices were selected based on their properties [6164]. The matrix materials were poly(methyl methacrylate), Mw ~120,000 by GPC, and polystyrene, Mw 35,000. Toluene was used for all experiments. 4,4′-bis(9-carbazolyl)benzophenone is commercially available from Hoffman Fine Chemicals (Victoria, Australia), while all other chemicals are commercially available from Sigma-Aldrich (St. Louis, MO, USA). Additional Details are included in the Supporting Information, S1 File.

Optical measurements.

Interrogation of the luminescent samples was conducted using our smartphone-based prototype device incorporating a custom printed circuit board able to pulse the UV LED contained on the board in response to a triggering event (the smartphone’s camera flash) (see Fig 2A). In earlier experiments, a Koolertron DSS Signal Generator Counter (60MHz High Precision Dual-channel Arbitrary Waveform Generator) was used to set the strobe frequency, waveform, and duty cycle of the excitation source. The signal generator was interfaced with a Zk-SJVA-4X 35W Step Up/Down Constant Current Converter Power Supply, and a XY-GMOS 10A MOSFET Motor driver and HIGH LOW trigger switch module, to control and power the excitation source. A Vishay 365nm LED (VLMU3510-365-130) was selected as the excitation source for this work. The effect of the LED’s rise and decay time was mitigated by the selection of a square waveform with a 50% duty cycle. Images were captured with the 26mm camera on iPhone model(s) 11 and 12 that were purchased from Apple Inc. (Cupertino, CA, USA). Additional Details are included in the Supporting Information, S1 File.

Image analysis.

ImageJ (Version: 2.1.0/1.53c, build 5f23140693) and the smartphone compatible, browser based ImageJ.JS (https://ij.imjoy.io) were used for the analysis of captured images. Analysis involved separating the captured RGB image into individual color channels. Once separated into individual color channels; the plot profile function was used to interrogate the grayscale image(s). Additional Details are included in the Supporting Information, S1 File.

Luminescent samples.

Solution-based samples (Fig 2B and 2C) comprised 0.1mM (1,10-phenanthroline)tris[4,4,4-trifluoro-1-(2-thienyl)-1,3-butanedionato]europium(III) and 0.0667mM N,N’-bis(1-hexylheptyl)perylene-3,4,9,10-tetracarboxybisimide dissolved in toluene.

The solid state sample, noted in Fig 1C, consisted of 1.0mM (1,10-phenanthroline)tris[4,4,4-trifluoro-1-(2-thienyl)-1,3-butanedionato]europium(III) and 0.25mg/ml poly(2,5-di(3’,7’-dimethyloctyl)phenylene-1,4-ethynylene) in a 0.166mM solution of poly(methyl methacrylate), dissolved in toluene and filtered (Whatman Puradisc 25 (Cat. No. 6747–2502, Lot. No. 17236201)). A Badger 105 Patriot airbrush (Franklin Park, IL, USA) was used to deposit approximately 3.72x10-4mL of solution per square centimeter. Dry nitrogen at a line pressure of 0.35 bar was used as the propellent with the airbrush held 15cm from the substrate.

thumbnail
Fig 1. Visualization of the CMOS rolling shutter’s potential to measure time-gated signals.

(A) Schematic representation of Rolling Shutter operation depicting the Exposure Time, Readout Time, and Reset Time. The duty cycle of a PWM modulated LED is also depicted (gray overlay). As illustrated, the rapid pulsing of the LED will only partially expose rows during the capture, resulting in a banded image. (B) Rolling Shutter based image of a pulsed LED captured using the 26mm camera of an iPhone 12. ISO 3200, Exposure 1/71429 s, f1.6. (C) A fast emitter, Poly(2,5-di(3’,7’-dimethyloctyl)phenylene-1,4-ethynylene), and slow emitter, Eu(tta)3phen, co-deposited on a solid support excited with a 365nm UV LED pulsed at 600.08 Hz. The image was captured using the 26mm camera of an iPhone 12. ISO 3,200, Exposure 1/71429 s, f1.6. (D) Analysis of On/Off transitions from the image shown in Fig 1C illustrating the presence of a fast emitter (blue) in the “on” band and slow emitter (red) in the “off” band.

https://doi.org/10.1371/journal.pone.0293740.g001

The item illustrated in Fig 3, is composed of three unique toluene-based formulations. The red emissive material’s formulation contained 1.0mM (1,10-phenanthroline)tris[4,4,4-trifluoro-1-(2-thienyl)-1,3-butanedionato]europium(III) and 0.166mM poly(methyl methacrylate). The green emissive material’s formulation contained 35.0mM terbium 2,2,6,6-tetramethyl-3,5-heptanedionate contained 0.166mM poly(methyl methacrylate). The yellow/green emissive material’s formulation contained 0.025mg/mL poly(9,9-dioctylfluorene-alt-benzothiadiazole) and 0.166mM poly(methyl methacrylate). After dissolving, the solution(s) were filtered (Whatman Puradisc 25 (Cat. No. 6747–2502, Lot. No. 17236201)). The materials were individually deposited onto cardstock via a Techcon TSR2401 Automated Gantry Robot with a TS5540-MS spray-coating valve. The Techcon was run at a rate of 100mm/second with a 50mm Z-axis offset. The machine’s pneumatic activation pressure was set to 5.5 bar with a feed pressure of 1.03 bar. Additional Details are included in the Supporting Information, S1 File.

Oxygen / Pressure sensing.

The sensing materials, as noted in Fig 4A, consisted of 1.90mM of 4,4′-bis(9-carbazolyl)benzophenone in a 1.43mM solution of polystyrene, dissolved into toluene. This formulation was deposited with a Badger 105 Patriot airbrush (Franklin Park, IL, USA) at a rate of 0.013mL/cm2 onto Avery Shipping Address Labels (manufacturers part no. 5135). Dry nitrogen with a line pressure of 0.25 bar was selected as the propellent, with an offset of 15cm between the airbrush and the selected substrate. Samples were dried under high vacuum for a minimum of 2h.

The testing setup consisted of an Edwards RV5 rotary vane vacuum pump, standard Swagelok fittings, and Chemglass glassware. An Omega battery powered digital pressure gauge (Measuring Range: -1.0 to 345.0 Millibar, Accuracy: ±0.5% FS, Stability: ±0.1% FS per year) was used to monitor and calibrate the system. The use of vacuum as a proxy for oxygen concentration has been previously demonstrated [6568]. Additional Details are included in the Supporting Information, S1 File.

Temperature sensing.

The temperature sensors were created by spin-coating onto DYMO Permanent Polyester Labels (Part number 1734523), using a KW-4A Spin Coater from Setcas LLC (San Diego, CA, USA), 1.0mM (1,10-phenanthroline)tris[4,4,4-trifluoro-1-(2-thienyl)-1,3-butanedionato]europium(III) dissolved in a 0.166mM solution of poly(methyl methacrylate) in toluene. After filtration (Whatman Puradisc 25 (Cat. No. 6747–2502, Lot. No. 17236201)), 70μl of solution was deposited onto a 24mm x 24mm label and spun at 3000rpm for 45 seconds.

The sensors (ten replicates) were affixed to a 3105-H24 aluminum alloy (thickness = 0.6858mm) plate and suspended inside a thermal chamber (TPS, Model No. TUJR-A-F4T, Temperature Range: -68°C to +180°C). Measurements were collected from approximately 15cm away under ambient and dark lighting conditions using the prototype device. The experimental conditions (-25°C to +100°C) were confirmed using a thermal imaging infrared camera (Teledyne FLIR, Model: FLIR E8). Additional Details are included in the Supporting Information, S1 File.

Results and discussion

To unlock the smartphone camera’s potential to measure time-gated signals, we started by examining the CMOS rolling shutter’s reproducibility and fidelity using a 365nm LED pulsed at 600.08 Hz during image capture. The resulting photograph, as shown in Fig 1B, exhibited the characteristic banding artifact—a visualization of the LED’s “on-off” cycle—produced by a rolling shutter sensor observing a pulsed LED reflecting off a surface. The symmetry of the band widths corresponds to the LED duty cycle (preset here to 50%), with the total number of “on-off” cycles directly tied to the pulse frequency.

Fig 1C shows a non-fluorescent substrate coated with two emissive dyes: a fast emitter with an emissive lifetime of less than a nanosecond, and a slow emitter with a microsecond-range emissive lifetime. Upon strobing with a 365nm LED, the resulting “on” band contains emission from both the fast (blue) and slow (red) emitters, while the “off” band only contains the slow (red) emitter’s emission. Pixel-based color analysis of the captured image validates these observations.

Using commercially available image analysis software, we can extract a wealth of data, as illustrated in Fig 1D the graph illustrates the “on” and “off” bands. When the pulsed 365nm LED is on, the fast-emitting dye is excited and emits blue light, but when the LED is off, the dye quickly relaxes; essentially following the LED’s duty cycle. In contrast, the slow emitter, due to its longer lifetime, does not mimic the LED’s duty cycle. Overlaying the different color channels with the LED’s status highlights this (Fig 1D). Curve-fitting the intensity profile of the slow-emitter yields its decay constant, aligning well with the reported literature value. To the best of our knowledge, this marks the first instance of time-delayed measurements executed via a smartphone.

This experiment also allowed us to identify which luminescent materials were a good fit for the system, given that the CMOS imager’s time resolution becomes the limiting factor when detecting fast emitters. Consequently, this work made use of phosphorescent and time-delayed fluorescent dyes, which display emissions in the micro- to millisecond range. Specifically, we focused on commercially available lanthanides [69, 70], thermally activated delayed fluorescent (TADF) organic materials [71, 72], and fluorescent dyes.

We then developed an integrated, field-deployable prototype modeled on a typical smartphone case (Fig 2A). This device allows the smartphone to work in concert with strobed LED excitation sources. The case makes use of the original equipment manufacturer (OEM) smartphone cameras (i), includes a generic solder pad for the LED excitation source (ii), houses the LED’s driver-board and power source (iii), and allows the device to function as a self-contained handheld unit.

thumbnail
Fig 2. The integrated smartphone-based prototype and its function as a low-cost reader.

(A) The integrated prototype illustrated with callouts for: i) the smartphone camera, ii) the LED excitation source, and iii) the case including rechargeable battery to power the LED. (B) A homogeneous mixture of fast emitter, N,N’-Bis(1-hexylheptyl)perylene-3,4,9,10-tetracarboxybisimide and slow emitter, Eu(tta)3phen dissolved in toluene. The image and subsequent analysis illustrate the system’s inability to differentiate between the fast and slow emitter when excited by constant 365nm illumination. (C) The same solution (as shown in 2b) interrogated with pulsed 365nm illumination. Both the image and analysis illustrate the system’s ability to deconvolute the fast emission (green/yellow) from the slow emission (red).

https://doi.org/10.1371/journal.pone.0293740.g002

The case design was optimized for the 26 mm camera with the LED’s position adjusted to align with the camera’s field of view. This setup enabled both luminescence and colorimetric images to be collected and analyzed via the readily available ImageJ software.

We then performed experiments to test the smartphone-based device’s ability to function as a low-cost reader. The results of these tests are shown in Fig 2B and 2C, comparing steady-state (constant illumination) and rolling shutter (pulsed illumination) images of non-aqueous solutions. The solution contained two different commercially available dyes with distinct emissive wavelengths and lifetimes, allowing us to compare steady-state fluorescence and lifetime-based measurements. The steady-state image illustrates the combined emission resulting from 365nm excitation, thereby preventing species differentiation in this complex mixture (Fig 2B). As a result, it is difficult to perform quantitative measurements or extract the decay constants.

In contrast, our device can differentiate the two species based on their lifetimes (Fig 2C). As illustrated, the emission from the fast emitter, N,N’-Bis(1-hexylheptyl)perylene-3,4,9,10-tetracarboxybisimide, directly correlates with the LED’s duty cycle. This confines the fast emission to the "on" band. Meanwhile, the slow emission from Eu(tta)3phen does not correlate with the device’s LED duty cycle and appears in the "off" band. Analysis of the emission allows for decay constant calculation and positive identification of the species.

Further vetting of our device’s function as a low-cost reader made use of lanthanide-based complexes. Lanthanide complexes are often used as security inks for products, documents, and currency due to their resistance to photobleaching, narrow emission profiles, and reduced self-quenching [7375]. The inks allow for the insertion of hidden information into spatial patterns via their emission wavelengths and luminescent lifetimes. Current methods for analyzing these inks include simple visual inspection (presence/absence) or more complex laboratory-based analysis. Existing approaches fail to fully exploit the potential of these materials and/or increase the cost of analysis, thereby limiting their commercial impact.

Our smartphone-based device addresses these issues by enabling both luminescence and lifetime analysis of lanthanide security inks. Its versatility was demonstrated by fabricating tags that were invisible to the naked eye, function as typical security tags under steady-state UV illumination and reveal hidden time-resolved patterns under pulsed UV illumination when analyzed by our device (Fig 3). This level of analysis was previously only possible with laboratory time-resolved instruments.

thumbnail
Fig 3. Luminescence and lifetime analysis of lanthanide security inks via the smartphone-based device.

(A) Geometric patterning of a fast emitter, Poly(9,9-dioctylfluorene-alt-benzothiadiazole) (yellow/green) and multiple slow emitters, Eu(tta)3phen (red) and Terbium 2,2,6,6-tetramethyl-3,5-heptanedionate (green). Interrogation under (i) ambient light, (ii) constant 365nm illumination, and (iii) strobed 365nm excitation with rolling shutter. (B) The same geometric pattern illustrated in Fig 3A combined with a QR code to enable multiple authentication techniques. Under (i) ambient light only the QR code is visible. Constant illumination (ii) by 365nm reveals both the QR code and the pattern’s fast emission. Figure (iii) demonstrates strobed 365nm excitation and imaging with rolling shutter.

https://doi.org/10.1371/journal.pone.0293740.g003

In addition to the analysis of lanthanide complexes, we vetted of our device’s ability to analyze TADF and phosphorescent materials for sensing applications. A well-known phenomenon in photochemistry is the quenching of long-lived excited states by molecular oxygen, enabling the quantitative measurement of oxygen concentrations based on the material’s emission [47, 7680]. However, leveraging such materials as oxygen sensors has been problematic due to the hardware required to analyze the emitting species. Our smartphone-based prototype offers a portable and field-deployable solution for reading oxygen-sensitive delayed emitters.

As demonstrated in Fig 4A, a well-known TADF material, 4,4′-bis(9-carbazolyl)benzophenone, was embedded in an oxygen-permeable matrix [57, 58], deposited on a substrate and exposed to various ambient oxygen concentrations by reducing the ambient pressure [6568]. As shown in panels i–v, the emission from this material was variably quenched based on the amount of oxygen present. Using our device, we could capture images obtained under different pressures/oxygen concentrations. Analysis of these images enables the calculation of the resultant intensity and decay rates, enabling non-invasive, line-of-sight oxygen sensing (Fig 4B and 4C). Additional testing vetted the device’s ability to function under different lighting conditions. As depicted in Fig 4D, we monitored the response of a temperature responsive material (exhibiting delayed emission) under ambient and dark conditions. In both cases the resulting lifetime data correlated with the ambient temperature (Fig 4E and 4F), further highlighting the smartphone-based prototype’s function as a portable and field-deployable solution.

thumbnail
Fig 4. Analysis of oxygen concentration and temperature response via the smartphone-based device.

(A) A Tyvek support coated with 4,4′-bis(9-carbazolyl)benzophenone in polystyrene. Sensing is illustrated in i–v where the tag has been exposed to different amounts of oxygen. (B) Emission intensity as a function of ambient pressure. (C) Decay rate as a function of ambient pressure. (D) Temperature response of Eu(tta)3phen in PMMA imaged under ambient and dark lighting conditions. (E) Emission intensity as a function of temperature imaged under ambient lighting. (F) Emission intensity as a function of temperature imaged under dark lighting.

https://doi.org/10.1371/journal.pone.0293740.g004

Conclusion

In conclusion, we have demonstrated that conventional smartphones can be leveraged to perform exquisite time-delayed measurements. Coupling the smartphone’s rolling shutter mechanism with materials that exhibit long-lived emissions and pulsed excitation sources, enables the differentiation of materials based on their emissive lifetimes.

We present here several examples, including images of pulsed LEDs and emitters with different luminescent properties, illustrating the versatility and sensitivity of the system. Importantly, we also demonstrate that the system can perform quantitative measurements, thereby yielding valuable information about the concentration of analytes present.

The simplicity, portability, and affordability of this approach make it a promising tool for many application areas and opens new possibilities within the field of imaging for both academia and industry. We hope that our work encourages further exploration of smartphone-based luminescence imaging, accelerates its adoption, and inspires innovation via increased access to analytical techniques previously restricted to laboratory settings.

Supporting information

S1 File. Its contains additional information about the chemicals, deposition and interrogation methods used.

https://doi.org/10.1371/journal.pone.0293740.s001

(DOCX)

References

  1. 1. Kitchin R., The Data Revolution: Big Data, Open Data, Data Infrastructures & Their Consequences. 1st ed. London: Sage Publications Ltd; 2014.
  2. 2. Webster M, Kumar V, Automated doctors: cellphones as diagnostic tools. Clin. Chem. 2012;58: 1607–1609. pmid:23104961
  3. 3. Chin CD, Cheung YK, Laksanasopin T, Modena MM, Chin SY, Sridhara AA, et al. Mobile device for disease diagnosis and data tracking in resource-limited settings, Clin. Chem. 2012;59: 629–640.
  4. 4. Yadav R, Yadav L, Laddha H, Agarwal H. Upsurgence of smartphone as an economical, portable, and consumer-friendly analytical device/interface platform for digital sensing of hazardous environmental ions. Trends in Environmental Analytical Chemistry. 2022;36: e00177.
  5. 5. Yetisen A, Martinez-hurtado JL, Garcia-Melendrez A, da Cruz Vasconcellos F, Lowe CR. A smartphone algorithm with interphone repeatability for the analysis of colorimetric tests. Sensors and Actuators B: Chemical. 2014;196: 156–160.
  6. 6. Walish J, Cox J, Boone J, Stone J, Henderson N, Maloney M, et al. Halo–a universal fluorescence reader based threat agent detection platform–a proof of concept study using SARS-CoV-2 assays. Frontiers in Public Health. 2022; pmid:35493369
  7. 7. McGoldrick LK, Halamek J, Recent advances in noninvasive biosensors for forensics, biometrics, and cybersecurity. Sensors. 2020;20: 5974. pmid:33105602
  8. 8. Shen Y, Wei Y, Zhu C, Cao J, Han D. Ratiometric fluorescent signal-driven smartphone-based portable sensors for onsite visual detection of food contaminates, Coordination Chemistry Reviews. 2022;458: 214442.
  9. 9. Kilic B, Dogan V, Kilic V, Kahyaoglu LN, Colorimetric food spoilage monitoring with carbon dot and UV light reinforced fish gelatin films using a smartphone application, International Journal of Biological Macromolecules. 2022;209: 1562–1572. pmid:35469948
  10. 10. Patel S, Shrivas K, Sinha D, Patle MKT, Yadav S, Thakur SS, et al. Smartphone-integrated printed-paper sensor designed for on-site determination of dimethoate pesticide in food samples. Food Chemistry. 2022; pmid:35183953
  11. 11. McCracken KE, Yoon J. Recent approaches for optical smartphone sensing in resource-limited settings: a brief review. Anal. Methods. 2016;8: 6591–6601.
  12. 12. Hunt B, Ruiz AJ, Pogue BW, Smartphone-based imaging systems for medical applications: a critical review. Journal of Biomedical Optics. 2021;26: 040902. pmid:33860648
  13. 13. McDonagh C, Burke CS, MacCraith BD. Optical chemical sensors. Chem Rev. 2008;108(2): 400–22. pmid:18229950
  14. 14. Orellana G, Moreno-Bondi MC. Frontiers in Chemical Sensors: Novel Principles and Techniques, 1st ed. New York: Springer Publishing Company; 2014.
  15. 15. Baldini F, Chester AN, Homola J, Martellucci S. Optical Chemical Sensors; NATO Science Series 224. 1st ed. New York: Springer Publishing Company; 2006.
  16. 16. Narayanaswamy R, Wolfbeis OS. Optical Sensors—Industrial, Environmental and Diagnostic Applications. 1st ed. New York: Springer Publishing Company; 2004.
  17. 17. Wolfbeis OS. Fiber-optic chemical sensors and biosensors. Anal. Chem. 2006;78: 3859. pmid:16771528
  18. 18. Sánchez-Barragán I, Costa-Fernández JM, Sanz-Medel A, Valledor M, Campo JC. Room-temperature phosphorescence (RTP) for optical sensing. TrAC Trends in Analytical Chemistry. 2006; https://doi.org/10.1016/j.trac.2006.07.009.
  19. 19. Kostov Y, Rao G. Low-cost gated system for monitoring phosphorescent lifetimes. Rev Sci Instrum. 2003;74(9): 4129–4133.
  20. 20. Di Nonno S, Ulber R. Portuino–a novel portable low-cost Arduino-based photo- and fluorimeter. Sensors. 2022;22: 7916. pmid:36298268
  21. 21. Zare RN. Interference effects in molecular fluorescence. Acc. Chem. Res. 1971;4: 361–367.
  22. 22. Li W, Zhang X, Hu X, Shu Y, Li Z, Huang X, et al. A smartphone-integrated ratiometric fluorescence sensor for visual detection of cadmium ions. Journal of Hazardous Materials. 2021; 408:124872. pmid:33387715
  23. 23. Haidekker MA, Theodorakis EA. Ratiometric mechanosensitive fluorescent dyes: design and applications. J. Mater. Chem. C, 2016;4: 2707–2718. pmid:27127631
  24. 24. Thoren D, Thoren P, Schmidhalter U. Influence of ambient light and temperature on laser-induced chlorophyll fluorescence measurements. Europ. J. Agronomy. 2010;32: 169–176.
  25. 25. Yang W, Chen S. Time-gated fluorescence imaging: Advances in technology and biological applications. Journal of Innovative Optical Health Sciences. 2020; 13:03;
  26. 26. Chib R, Requena S, Mummert M, Strzhemechny YM, Gryczynski I, Borejdo J, et al. Fluorescence lifetime imaging with time-gated detection of hyaluronidase using a long lifetime azadioxatriangulenium (ADOTA) fluorophore. Meth. Appl. Fluorescence, 2016; 4(4): 047001. pmid:28192308
  27. 27. Connally R. A device for gated autosynchronous luminescence detection. Anal. Chem. 2011; 83(12): 4782–4787. pmid:21553867
  28. 28. Bünzli JCG. Luminescence Bioimaging with Lanthanide Complexes. 2014, 125–196. https://doi.org/10.1002/9781118682760.ch04.
  29. 29. Requejo-Isidro J, McGinty J, Munro I, Elson DS, Galletly NP, Lever MJ, et al. High-speed wide-field time-gated endoscopic fluorescence-lifetime imaging. Opt. Lett. 2004; 29: 2249–2251. pmid:15524370
  30. 30. Soloviev VY, Tahir KB, McGinty J, Elson DS, Neil MAA, French PMW, et al. Fluorescence lifetime imaging by using time-gated data acquisition. Appl. Opt. 2007; 46: 7384–7391. pmid:17952172
  31. 31. Mei E, et al. A line scanning confocal fluorescent microscope using a CMOS rolling shutter as an adjustable aperture. J Microsc. 2012; 247(3): 269–76. pmid:22906014
  32. 32. Hanaoka K, Kikuchi K, Kobayashi S, Nagano T. Time-Resolved Long-Lived Luminescence Imaging Method Employing Luminescent Lanthanide Probes with a New Microscopy System. Journal of the American Chemical Society 2007 129 (44): 13502–13509, pmid:17927176
  33. 33. Jovin TM, Marriott G, Clegg RM, Arndt-Jovin DJ. Photophysical processes exploited in digital imaging microscopy: Fluorescence resonance energy transfer and delayed luminescence. Berichte der Bunsengesellschaft für physikalische Chemie. 1989; 93: 387–391.
  34. 34. Jovin TM, Arndt-Jovin DJ. Luminescence Digital Imaging Microscopy. Annual Review of Biophysics and Biophysical Chemistry 1989; 18:1: 271–308.
  35. 35. Jin D, Connally R, Piper J. Practical time-gated luminescence flow cytometry. II: Experimental evaluation using UV LED excitation. Cytometry. 2007; 71A: 797–808. pmid:17868086
  36. 36. Gaft M, Panczer G, Reisfeld R, Uspensky E. Laser-induced time-resolved luminescence as a tool for rare-earth element identification in minerals. Physics and Chemistrys of Minerals. 2001;28: 347–363.
  37. 37. Draxler S, Lippitsch ME. Lifetime-based sensing: influence of the microenvironment. Anal. Chem. 1996;68: 753–757.
  38. 38. Lakowicz JR. Principles of Fluorescence Spectroscopy. 1st ed. New York: Plenum Press; 1983.
  39. 39. Bergua JF, Alvarez-Diduk R, Idili A, Parolo C, Maymo M, Hu L, et al. Low-cost, user friendly, all-integrated smartphone-based microplate reader for optical-based biological and chemical analyses. Anal. Chem. 2022;94: 1271–1285. pmid:34979088
  40. 40. McGown LB. Fluorescence Lifetime Filtering. Anal. Chem. 1989;61(14): 839A–847A.
  41. 41. Shen L, Hagen JA, Papautsky I. Point-of-care colorimetric detection with a smartphone. Lab Chip. 2012;12: 4240–4243. pmid:22996728
  42. 42. Al-Rawhani MA, et al. A Colorimetric CMOS-Based Platform for Rapid Total Serum Cholesterol Quantification. IEEE Sensors Journal. 2017;17(2): 240–247.
  43. 43. Palekar SD, Kalambe J, Patrikar RM. Biochemical Blood Sensing Platform with CMOS Image Sensor and Software-Based Wavelength Filter. IEEE Sensors Journal. 2022;22(22): 21753–21760.
  44. 44. Priye A, Ball CS, Meager RJ. Colorimetric-luminance readout for quantitative analysis of fluorescence signals with a smartphone CMOS sensor. Anal. Chem. 2018;90: 12385–12489. pmid:30272954
  45. 45. Li Z, Askim JR, Suslick SK. The optoelectronic nose: Colorimetric and fluorescent sensor arrays. Chem. Rev. 2019;119: 231–292.
  46. 46. Seo D, Oh S, Shin S, Lee M, Hwang Y, Seo S. Smartphone compatible on-site fluorescence analyzer for spilled crude oil based on CMOS image sensor. Sensors and Actuators B: Chemical. 2019;289: 93–99.
  47. 47. Kalyanaraman B, Darley-Usmar V, Davies KJA, Dennery PA, Jay Forman H, Grisham MB, et al. Measuring reactive oxygen and nitrogen species with fluorescent probes: challenges and limitations. Free Radical Biology and Medicine. 2012;52(1): 1–6. pmid:22027063
  48. 48. Han X, Song E, Zhou B, Zhang Q. Color tunable upconversion luminescent perovskite fluoride with long-/short-lived emission toward multiple anti-counterfeiting. J. Mater. Chem. C. 2019;7: 8226–8235.
  49. 49. Katumo N, Li K, Richards BS, Howard IA. Dual-color dynamic anti-counterfeiting labels with persistent emission after visible excitation allowing smartphone authentication. Scientific Reports. 2022;12: 2100. pmid:35136113
  50. 50. Paterson AS, Raja B, Mandadi V, Townsend B, Lee M, Buell A, et al. A low-cost smartphone-based platform for highly sensitive point-of-care testing with persistent luminescent phosphors. Lab Chip. 2017;17: 1051–1059. pmid:28154873
  51. 51. Provakar P, Arabinda M, Tapas M. True-photonic strategy for chemically encrypted two-factor authentication with unlimited chemical passwords based on cascade energy transfer under a single wavelength excitation. ACS Applied Electronic Materials. 2021;3(8): 3588–3598.
  52. 52. Xiong B, Fang Q. Luminescence lifetime imaging using a cellphone camera with an electronic rolling shutter. Optics Letters. 2020;45(1): 81–84.
  53. 53. Blahnik V, Schindelbeck O. Smartphone imaging technology and its applications. Adv. Optical Technologies. 2021;10: 145–232.
  54. 54. Köhler S, Lovisotto G, Birnbach S, Baker R, Martinovic I. They see me rollin’: Inherent vulnerability of the rolling shutter in CMOS image Sensors. ACSAC ‘21: Annual Computer Security Applications Conference (Association for Computer Machinery). https://doi.org/10.48550/arXiv.2101.10011.
  55. 55. Kuroda T, Essential Principles of Image Sensors, 1st ed. Boca Raton: CRC Press: 2015.
  56. 56. Pan Z, Lang T, Li Z, Chen G, Wang A. A Study of Optical Tag Detection Using Rolling Shutter Based Visible Light Communications. 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA, 2019:.
  57. 57. Zhao Y, Li Y, Guo S, Li T. Measuring the angular velocity of a propeller with a video camera using electronic rolling shutter. Journal of Sensors.
  58. 58. Cao M, Zhong Z, Wang J, Zheng Y, Yang Y. Learning adaptive warping for real-world rolling shutter correction, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022: https://doi.org/10.48550/arXiv.2204.13886.
  59. 59. Swager TM, Cox JR, Deans R. System and method for sensing using consumer electronic devices. US20210116376A1 (patent) 2019.
  60. 60. Swager TM, Deans R, Cox JR, Takle JN, Davis G, Wheeler A, et al. Luminescence imaging for sensing. PCT/US20/56091 (patent) 2019.
  61. 61. Koren K, Hutter L, Enko B, Pein A, Borisov SM, Klimant I. Tuning the dynamic range and sensitivity of optical oxygen-sensors employing differently substituted polystyrene-derivatives. Sensors and Actuators B. 2013;176: 344–350.
  62. 62. Brandrup J, Immergut EH, Grulke EA. Polymer Handbook, 2 Volumes Set. 4th ed, New York: Wiley; 2003.
  63. 63. Lalova A, Todorov R. Optical properties of thin PMMA films for sensor applications. Bulgarian Chemical Communications. 2015;47: 29–34.
  64. 64. Zabihi F, Eslamian M. Characteristics of thin films fabricated by spray coating on rough and permeable paper substrates. Journal of Coatings Technology and Research. 2015;12: 489–503.
  65. 65. Marquardt N., Introduction to the principles of vacuum physics. CAS–CERN Accelerator School: Vacuum Technology. 1–24,
  66. 66. Berman A., Vacuum Engineering Calculations, Formulas, and Solved Exercises, 1st ed. Cambridge, MA: Academic Press, Inc.; 1992.
  67. 67. Lafferty JM (ed), Foundations of Vacuum Science and Technology, 1st ed. New York, NY: John Wiley and Sons Inc,: 1988.
  68. 68. Atkins P, De Paula J, Keeler J. Atkins’ Physical Chemistry. 11th ed. London, England: Oxford University Press; 2017.
  69. 69. Choi PR, Park HH, Koo E. Synthesis and characterization of luminescent Eu(TTA)3phen in a Poly(ethylene oxide) matrix for detecting traces of water. Electronic Materials. 2013;42: 927–930.
  70. 70. Wang X, Wolfbeis OS, Meier RJ. Luminescent probes and sensors for temperature. Chem. Soc. Rev. 2020;42: 7834–7869.
  71. 71. Paisley NR, Tonge CM, Hudson ZM. Stimuli-responsive thermally activated delayed fluorescence in polymer nanoparticles and thin films: applications in chemical sensing and imaging. Front. Chem.. 2020;8: 229. pmid:32328478
  72. 72. Steinegger A, Klimant I, Borisov SM. Purely organic dyes with thermally activated delayed fluorescence–a versatile class of indicators for optical temperature sensing. Advanced Optical Materials. 2017;5: 1700372.
  73. 73. Gao M, Li J, Xia D, Jiang L, Peng N, Zhao S, et al. Lanthanides-based security inks with reversible luminescent switching and self-healing properties for advanced anti-counterfeiting. Journal of Molecular Liquids. 2022;350: 118559.
  74. 74. MacKenzie LE, Pal R. Circularly polarized lanthanide luminescence for advanced security inks. Nature Reviews Chemistry. 2021;5: 109–124. pmid:37117607
  75. 75. Kumar P, Singh S, Gupta BK. Future prospects of luminescent nanomaterial based security inks: from synthesis to anti-counterfeiting applications. Nanoscale. 2016;30: 14297–14340. pmid:27424665
  76. 76. Wang XD, Wolfbeis OS. Optical methods for sensing and imaging oxygen: materials, spectroscopies, and applications. Chem. Soc. Rev. 2014;43: pmid:24638858.
  77. 77. Banerjee S, Kelly C, Kerry JP, Papkovsky DB. High throughput non-destructive assessment of quality and safety of packaged food products using phosphorescent oxygen sensors. Trends in Food Science & Technology. 2016; 50: 85–102,
  78. 78. Cartwright LC. Measurement and Analysis of Gases in Vacuum-Packed Food Containers. Industrial & Engineering Chemistry Analytical Edition 1946; 18 (12): 779–785, DOI:
  79. 79. Yusufu D, Mills A. A general-purpose colourimetric air pressure indicator. Sensors and Actuators B: Chemical. 2020; 305: 127425,
  80. 80. Smiddy M, Papkovskaia N, Papkovsky DB, Kerry JP. Use of oxygen sensors for the non-destructive measurement of the oxygen content in modified atmosphere and vacuum packs of cooked chicken patties; impact of oxygen content on lipid oxidation. Food Research International. 2002; 35 (6): 577–584,