We present FLIMngo, a novel network for predicting fluorescence lifetimes from raw TCSPC-FLIM data. Our model is based on the YOLOv5 architecture, which has been adapted for pixel-wise regression tasks.

Deep learning for fluorescence lifetime predictions enables high-throughput in vivo imaging
Sofia Kapsiani, Nino F Läubli, Edward N. Ward, Ana Fernandez-Villegas, Bismoy Mazumder, Clemens F. Kaminski, Gabriele S. Kaminski Schierle
Molecular Neuroscience Group and Laser Analytics Group (University of Cambridge)

[Manuscript(bioRxiv)] [Supplementary Information] [Citation]

git clone https://github.com/SofiaKapsiani/FLIMngo.git
cd FLIMngo

# Create and activate a Conda environment
conda create --name flimngo_env python=3.9 -y
conda activate flimngo_env

# Install dependencies
pip install -r requirements.txt

Predictions can be made using the pretrained model file, flimngo_pretrained_v13102024.pth.

• Bin Width (ns): bin_width of time channels in nanoseconds for the raw data.
• X, Y Dimensions: Input data must have equal x and y dimensions (e.g., 256 × 256).
• Time Dimensions: The model currently only accepts raw data with 256 time dimensions.
  • For data that do not match this requirement, refer to predict_diff_time_dimensions.ipynb in demo_notebooks for a method to artificially expand/compress time dimensions.

• Normalisation: Time dimensions should be normalised to a range between 0 and 1.
  • See preprocessing steps in demo_notebooks.
• Background Masking: The background should be masked using either:
  • Intensity thresholding
  • Manual intensity masks (refer to predict_celegans_dynamic.ipynb in demo_notebooks for details).

Please note the model has been optimised for data collected with IRFs ranging from 100-400 ps.

FLIMngo maintains high prediction accuracy even for FLIM data with fluorescence decay curves containing as few as 10 photon counts.

• predict_simulated.ipynb: Evaluates performance on synthetic FLIM data with varying photon counts per pixel.
• predict_reduced_photon_counts.ipynb: Demonstrates performance on images from different experiments with at least 100 photon counts per pixel, as well as the same images with artificially reduced photon counts (10–100 photons per pixel).
• predict_diff_time_dimensions.ipynb: Example of predicting lifetimes from input data that do not have 256 time dimensions, with a method for time dimension adjustment.
• predict_celegans_dynamic.ipynb: Predicting lifetimes from dynamic, non-anesthetised C. elegans.

The fluorescence intensity images shown in (a) are taken from the Human Protein Atlas (HPA) dataset (Kaggle HPA Single-Cell Image Classification).

HPA images consist of RGBY color channels, representing:

• R (Red) – Microtubules
• G (Green) – Protein
• B (Blue) – Nucleus
• Y (Yellow) – Endoplasmic Reticulum (ER)

To generate simulated FLIM data, run the notebooks found in data_simulation directory in the following order:

1. notebook1_cropp_imgs.ipynb

   • Applies a sliding window approach to extract 256×256 pixel sub-images (x, y) from the HPA fluorescence intensity images, as shown in (a).

2. notebook2_irf_simulation.ipynb

   • Generates a dataset containing both experimentally acquired and simulated instrument response functions (IRFs).

3. notebook3_lifetime_simulation.ipynb

   • Simulates 3D FLIM data by assigning a fluorescence lifetime range to each HPA color channel, as illustrated in (b).
   • Perlin noise (example in (c)) is used to determine the fractional contribution of the first color channel to each pixel.
   • For each pixel, fluorescence decay curves are simulated using the following equation:

   $$ y(t) = IRF \otimes \sum_{i=1}^{n} \left( a_i e^{-t/\tau_i} \right) + \text{noise} \tag{1} $$

   where:

• $$IRF$$ represents the instrument response function.
• $$n$$ is the number of lifetime components (i.e., the number of color channels contributing to the pixel).
• $$a_i$$ and $$\tau_i$$ are the fractional contribution and fluorescence lifetime of each color channel at a given pixel, respectively.
• $$\text{noise}$$ accounts for the Poisson noise typically encountered in TCSPC systems.
• $$\otimes$$ denotes the convolution between the decay curve and the IRF.

For further details, please refer to the Materials and Methods section of our manuscript.

If you found FLIMngo helpful, please consider citing our work! 😊

@article {Kapsiani2025.02.20.639036,
	author = {Kapsiani, Sofia and L{\"a}ubli, Nino F and Ward, Edward N and Fernandez-Villegas, Ana and Mazumder, Bismoy and Kaminski, Clemens F and Kaminski Schierle, Gabriele S},
	title = {Deep learning for fluorescence lifetime predictions enables high-throughput in vivo imaging},
	elocation-id = {2025.02.20.639036},
	year = {2025},
	doi = {10.1101/2025.02.20.639036},
	URL = {https://www.biorxiv.org/content/early/2025/02/26/2025.02.20.639036},
	eprint = {https://www.biorxiv.org/content/early/2025/02/26/2025.02.20.639036.full.pdf},
	journal = {bioRxiv}
}