Blind and Reference-free Fluorescence Lifetime Estimation

via Consumer Time-of-Flight Sensors

FAQ: Frequently Asked Questions
  1. (1)What are you trying to do?
    Fluorescence lifetime imaging (FLI) is a well established imaging parameter that finds important applications across several areas of life sciences. Examples include DNA sequencing, malignant tumor detection and super-resolution microscopy. Generally, FLI is performed using sophisticated electro-optical instruments which are expensive and cost in the range of several thousands of dollars. In our work, we show that it is possible to trade-off the precision in electro-optical instruments with sophistication in computational methods used for lifetime estimation purposes. To this end, we repurpose low-cost, time-of-flight or ToF sensors (such as the Microsoft Kinect) for FLI.

  2. (2)What is a Time-of-Flight or ToF camera?
    Unlike conventional digital cameras that capture two dimensional information of a scene, ToF cameras acquire the three-dimensional information. This this is accomplished by capturing two photographs per exposure. The first photograph is the conventional digital image of the scene that is also known as the amplitude image. However, the second photograph—known as the depth or the phase image—is special in that it computes the range or the depth of the scene per pixel. More precisely, each pixel of the depth image records the corresponding depth of the object in the scene. Combination of the amplitude/depth image pair results in the three dimensional image of the scene.

  3. (3)How does the Time-of-Flight camera work? 
    As the name suggests, ToF cameras are based on the time-of-flight principle—the amount of time it takes for the light to reflect back from a scene. Since time is proportional to distance, knowing the time-delay amounts to knowing the distance. This can be thought of SONAR with light.

  4. (4)How can the Time-of-Flight sensor be used for Fluorescence Lifetime Imaging?
    One of the predominant methods/tools linked with fluorescence lifetime imaging is the Fluorescence-lifetime imaging microscopy or FLIM. Typically, FLIM is categorized into to two complementary modes: time-domain and frequency domain. In time-domain FLIM, an impulse-like excitation pulse probes the fluorescent sample, and the time-resolved reflection is used to calculate lifetimes. In frequency-domain, the sample is excited with (sinusoidal) intensity modulated light, and the measured phase shift of the reflected signal at the same modulation frequency encodes the lifetime. As is shown in our paper, by drawing parallels between FLIM and ToF imaging systems, we show that ToF images functionally encode lifetime information which may be estimated using algorithms.

  5. (5)Are there some advantages to using Time-of-Flight sensor for Fluorescence Lifetime Imaging? 
    There are three main advantaged of using ToF sensors for Fluorescence Lifetime Imaging:

  6. (6)Can your approach be used in microscopy mode as a substitute to FLIM?
    A salient feature about the technique is that it is a local, per-pixel calculation, so that the lateral scale of the problem does not influence the technique. Thus, our proof-of-principle demonstration can be extended to integration with microscopy, both wide-field and point-scanning techniques, provided there is no pixel crosstalk. In fact, our current system is not aberration-corrected, and the resulting model mismatch makes reconstruction more challenging. A well-calibrated microscopy setup should alleviate this mismatch and improve results.

  7. (7)Many interesting life-science applications require estimation of lifetimes within few nanosecond range.
    Can this be done with Time-of-Flight sensors?
    For biological imaging, lifetimes are on the order of nanoseconds. Recent work suggests that current ToF sensors (with bandwidths of tens of MHz) are optimal for recovering lifetimes under 5 ns. For example, a 3 ns, lifetime is optimally estimated by a 30 MHz signal. Further, a similar ToF setup estimated lifetimes on the order of 4 ns. The important difference here is that the present approach simultaneously estimates lifetime and distance based on the same measurement. Our extensive study shows that with ~ 80 MHz bandwidth, ToF sensors can simultaneously estimate distances as well as lifetimes within few nanoseconds.

  8. (8)Compared to TCSPC based systems which operate at picosecond time resolution, time-domain Time-of-Flight sensors operate at a much coarser time scale of nanoseconds. Is this a limitation of the system?
    What singles out our method is the fact that our time-domain method is not constrained by the high time resolution requirement of time-domain FLIM, which requires illumination to be an impulse excitation. For example, TCSPC based systems can attain pulses as narrow as 50 to 60 ps in sense of full--width at half maximum intensity (FWHM). In contrast, the FWHM of the pulse in our demonstration is approximately 11 ns.

    Although our demonstration here operates at a lower time scale than what is typical in practice, this is not a fundamental limitation of the method. The reason is that we compensate for lower time resolution by utilizing a computationally different method of inversion. Indeed, appropriate modeling and prior information lend themselves naturally to ToF sensing and offer a path toward super-resolution.

  9. (9)Can you comment on the experimental and computational precision of ToF based Fluorescence Lifetime Imaging?
    The current optical ToF technology allows for range estimation in millimeter precision. The idea of lifetime and range imaging is new and the exact bounds on precision are under investigation. That said, the variation in estimated lifetime is attributed to sample inhomogeneity, nonuniform lighting, and potential model mismatch from lens aberration.

  1. Cost Effectivity
    ToF sensors are cheap and hence, this massively reduces the cost of the FLI system. For example the Microsoft Kinect costs about $100 compared to the ~ $175,200 of Leica’s TCS SMD FLIM system. Our custom made sensor together with the experimental setup costs with in $1200.

  2. Calibration-free Solution
    Existing FLI systems assume the knowledge of sample’s location/depth with respect to the sensor and this information must be calibrated for lifetime estimation purposes. Our method not only estimates the lifetime but also the location/depth of the sample relative to the ToF sensor.

  3. Blind Approach
    Existing FLI systems, time-domain or frequency domain, assume the knowledge of illumination signal. This is not the case in our method. Hence, our method is blind in that we do not assume any knowledge of the illumination signal.

Optica, OSA


FLI-ToF : Low-cost Lifetime Imaging
 Ayush Bhandari*
Christopher Barsi
Ramesh Raskar