作者:Hatzakis, Nikos S.
All data are available via the University of Copenhagen repository at https://erda.ku.dk/archives/804ea1ea88f340b79ada3e57141a6d6e/published-archive.html (ref. 67). All biological materials have no restrictions and are available upon reasonable request. We refer to the original publications for the additional experimental data used in this work: refs. 10,26,68.
A minimal repository of code is available at https://erda.ku.dk/archives/752e4b0695c0dd16ec3c1a130f6ac70b/published-archive.html (ref. 69). The repository of code and models is available at https://erda.ku.dk/archives/4c5adaaacc5c867f6450bcf89ec55a45/published-archive.html (ref. 70). Both are under a CC BY-NC-ND 4.0 DEED license. In addition, upon publication, the code will also be freely available on GitHub.
Cocucci, E., Aguet, F., Boulant, S. & Kirchhausen, T. The first five seconds in the life of a clathrin-coated pit. Cell 150, 495–507 (2012).
Johnson, C., Exell, J., Lin, Y., Aguilar, J. & Welsher, K. D. Capturing the start point of the virus-cell interaction with high-speed 3D single-virus tracking. Nat. Methods 19, 1642–1652 (2022).
Liu, T.-L. et al. Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms. Science 360, eaaq1392 (2018).
Thomsen, R. P. et al. A large size-selective DNA nanopore with sensing applications. Nat. Commun. 10, 5655 (2019).
Aguet, F. et al. Membrane dynamics of dividing cells imaged by lattice light-sheet microscopy. Mol. Biol. Cell 27, 3418–3435 (2016).
Moses, M. E. et al. Single-molecule study of Thermomyces lanuginosus lipase in a detergency application system reveals diffusion pattern remodeling by surfactants and calcium. ACS Appl. Mater. Interfaces 13, 33704–33712 (2021).
Jensen, S. B. et al. Biased cytochrome P450-mediated metabolism via small-molecule ligands binding P450 oxidoreductase. Nat. Commun. 12, 2260 (2021).
Gabriele, M. et al. Dynamics of CTCF- and cohesin-mediated chromatin looping revealed by live-cell imaging. Science 376, 496–501 (2022).
Jaqaman, K. et al. Robust single-particle tracking in live-cell time-lapse sequences. Nat. Methods 5, 695–702 (2008).
Wan, F. et al. Ultrasmall TPGS-PLGA hybrid nanoparticles for site-specific delivery of antibiotics into Pseudomonas aeruginosa biofilms in lungs. ACS Appl. Mater. Interfaces 12, 380–389 (2020).
Gal, N., Lechtman-Goldstein, D. & Weihs, D. Particle tracking in living cells: a review of the mean square displacement method and beyond. Rheol. Acta 52, 425–443 (2013).
Arcizet, D., Meier, B., Sackmann, E., Rädler, J. O. & Heinrich, D. Temporal analysis of active and passive transport in living cells. Phys. Rev. Lett. 101, 248103 (2008).
Pinholt, H. D., Bohr, S. S.-R., Iversen, J. F., Boomsma, W. & Hatzakis, N. S. Single-particle diffusional fingerprinting: a machine-learning framework for quantitative analysis of heterogeneous diffusion. Proc. Natl Acad. Sci. USA 118, e2104624118 (2021).
Kowalek, P., Loch-Olszewska, H. & Szwabiński, J. Classification of diffusion modes in single-particle tracking data: feature-based versus deep-learning approach. Phys. Rev. E 100, 032410 (2019).
Benning, N. A. et al. Dimensional reduction for single-molecule imaging of DNA and nucleosome condensation by polyamines, HP1α and Ki-67. J. Phys. Chem. B 127, 1922–1931 (2023).
Vega, A. R., Freeman, S. A., Grinstein, S. & Jaqaman, K. Multistep track segmentation and motion classification for transient mobility analysis. Biophys. J. 114, 1018–1025 (2018).
Monnier, N. et al. Inferring transient particle transport dynamics in live cells. Nat. Methods 12, 838–840 (2015).
Persson, F., Lindén, M., Unoson, C. & Elf, J. Extracting intracellular diffusive states and transition rates from single-molecule tracking data. Nat. Methods 10, 265–269 (2013).
Chen, Z., Geffroy, L. & Biteen, J. S. NOBIAS: analyzing anomalous diffusion in single-molecule tracks with nonparametric Bayesian inference. Front. Bioinform. 1, 742073 (2021).
Arts, M., Smal, I., Paul, M. W., Wyman, C. & Meijering, E. Particle mobility analysis using deep learning and the moment scaling spectrum. Sci. Rep. 9, 17160 (2019).
Vink, J. N. A., Brouns, S. J. J. & Hohlbein, J. Extracting transition rates in particle tracking using analytical diffusion distribution analysis. Biophys. J. 119, 1970–1983 (2020).
Martens, K. J. A. et al. Visualisation of dCas9 target search in vivo using an open-microscopy framework. Nat. Commun. 10, 3552 (2019).
Karslake, J. D. et al. SMAUG: analyzing single-molecule tracks with nonparametric Bayesian statistics. Methods 193, 16–26 (2021).
Simon, F., Tinevez, J.-Y. & van Teeffelen, S. ExTrack characterizes transition kinetics and diffusion in noisy single-particle tracks. J. Cell Biol. 222, e202208059 (2023).
Momboisse, F. et al. Tracking receptor motions at the plasma membrane reveals distinct effects of ligands on CCR5 dynamics depending on its dimerization status. eLife 11, e76281 (2022).
Hansen, A. S. et al. Robust model-based analysis of single-particle tracking experiments with Spot-On. eLife 7, e33125 (2018).
Martens, K. J. A., Turkowyd, B., Hohlbein, J. & Endesfelder, U. Temporal analysis of relative distances (TARDIS) is a robust, parameter-free alternative to single-particle tracking. Nat. Methods https://doi.org/10.1038/s41592-023-02149-7 (2024).
Muñoz-Gil, G. et al. Objective comparison of methods to decode anomalous diffusion. Nat. Commun. 12, 6253 (2021).
You, B. & Yang, G. Attention-based LSTM for motion switching detection of particles in living cells. In 2021 International Joint Conference on Neural Networks (IJCNN) 1–6 (IEEE, 2021); https://doi.org/10.1109/IJCNN52387.2021.9533629
Dosset, P. et al. Automatic detection of diffusion modes within biological membranes using back-propagation neural network. BMC Bioinformatics 17, 197 (2016).
Wagner, T., Kroll, A., Haramagatti, C. R., Lipinski, H.-G. & Wiemann, M. Classification and segmentation of nanoparticle diffusion trajectories in cellular micro environments. PLoS ONE 12, e0170165 (2017).
Granik, N. et al. Single-particle diffusion characterization by deep learning. Biophys. J. 117, 185–192 (2019).
Simon, F., Weiss, L. E. & van Teeffelen, S. A guide to single-particle tracking. Nat. Rev. Methods Prim. 4, 66 (2024).
Qu, X. et al. Semantic segmentation of anomalous diffusion using deep convolutional networks. Phys. Rev. Res. 6, 013054 (2024).
Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
Thomsen, J. et al. DeepFRET, a software for rapid and automated single-molecule FRET data classification using deep learning. eLife 9, e60404 (2020).
Malle, M. G. et al. Single-particle combinatorial multiplexed liposome fusion mediated by DNA. Nat. Chem. 14, 558–565 (2022).
Levet, F. et al. SR-Tesseler: a method to segment and quantify localization-based super-resolution microscopy data. Nat. Methods 12, 1065–1071 (2015).
Kim, H. K. et al. SpCas9 activity prediction by DeepSpCas9, a deep learning-based model with high generalization performance. Sci. Adv. 5, eaax9249 (2019).
Wong, F. et al. Discovery of a structural class of antibiotics with explainable deep learning. Nature 626, 177–185 (2024).
Dunn, K. W., Kamocka, M. M. & McDonald, J. H. A practical guide to evaluating colocalization in biological microscopy. Am. J. Physiol., Cell Physiol. 300, C723–C742 (2011).
Merino Urteaga, R. & Ha, T. Mind your tag in single-molecule measurements. Cell Rep. Methods 3, 100623 (2023).
Yin, X.-X., Sun, L., Fu, Y., Lu, R. & Zhang, Y. U-Net-based medical image segmentation. J. Healthc. Eng. 2022, 4189781 (2022).
Ruthardt, N., Lamb, D. C. & Bräuchle, C. Single-particle tracking as a quantitative microscopy-based approach to unravel cell entry mechanisms of viruses and pharmaceutical nanoparticles. Mol. Ther. 19, 1199–1211 (2011).
Guo, C., Pleiss, G., Sun, Y. & Weinberger, K. Q. On calibration of modern neural networks. In Proc. 4th International Conference on Machine Learning 1321–1330 (JMLR, 2017).
Michalet, X. Mean square displacement analysis of single-particle trajectories with localization error: Brownian motion in an isotropic medium. Phys. Rev. E 82, 041914 (2010).
Slator, P. J., Cairo, C. W. & Burroughs, N. J. Detection of diffusion heterogeneity in single particle tracking trajectories using a hidden Markov model with measurement noise propagation. PLoS ONE 10, e0140759 (2015).
Abdelhakim, A. H. et al. Structural correlates of rotavirus cell entry. PLoS Pathog. 10, e1004355 (2014).
Salgado, E. N., Garcia Rodriguez, B., Narayanaswamy, N., Krishnan, Y. & Harrison, S. C. Visualization of calcium ion loss from rotavirus during cell entry. J. Virol. 92, e01327-18 (2018).
Aoki, S. T. et al. Cross-linking of rotavirus outer capsid protein VP7 by antibodies or disulfides inhibits viral entry. J. Virol. 85, 10509–10517 (2011).
Rink, J., Ghigo, E., Kalaidzidis, Y. & Zerial, M. Rab conversion as a mechanism of progression from early to late endosomes. Cell 122, 735–749 (2005).
Piper, R. C. & Katzmann, D. J. Biogenesis and function of multivesicular bodies. Annu. Rev. Cell Dev. Biol. 23, 519–547 (2007).
Collinet, C. et al. Systems survey of endocytosis by multiparametric image analysis. Nature 464, 243–249 (2010).
Cocucci, E., Gaudin, R. & Kirchhausen, T. Dynamin recruitment and membrane scission at the neck of a clathrin-coated pit. Mol. Biol. Cell 25, 3595–3609 (2014).
Allan, D., Caswell, T., Keim, N. & van der Wel, C. Trackpy: Trackpy v0.3.2. Zenodo https://doi.org/10.5281/zenodo.60550 (2016).
Mizrak, A. et al. Single-molecule analysis of protein targeting from the endoplasmic reticulum to lipid droplets. Preprint at bioRxiv https://doi.org/10.1101/2024.08.27.610018 (2024).
Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020).
Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) Vol. 9351 (eds Navab, N., Hornegger, J., Wells, W. M. & Frangi, A. F.) 234–241 (Springer, 2015).
Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. Optuna: a next-generation hyperparameter optimization framework. In Proc. 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (eds Teredesai, V. K. et al.) 2623–2631 (ACM Press, 2019); https://doi.org/10.1145/3292500.3330701
Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Chen, B.-C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).
Kang, Y.-L. et al. Inhibition of PIKfyve kinase prevents infection by Zaire ebolavirus and SARS-CoV-2. Proc. Natl Acad. Sci. USA 117, 20803–20813 (2020).
Bohr, F. et al. Enhanced hexamerization of insulin via assembly pathway rerouting revealed by single particle studies. Commun. Biol. 6, 178 (2023).
Østergaard, M., Mishra, N. K. & Jensen, K. J. The ABC of insulin: the organic chemistry of a small protein. Chem. Eur. J. 26, 8341–8357 (2020).
He, K. et al. Dynamics of phosphoinositide conversion in clathrin-mediated endocytic traffic. Nature 552, 410–414 (2017).
Kæstel-Hansen, J. DeepSPT data and models. University of Copenhagen https://doi.org/10.17894/ucph.75da99a5-f7f1-44e7-bb6e-3fcc97bf0a15 (2024).
Bohr, S. S.-R. et al. Direct observation of Thermomyces lanuginosus lipase diffusional states by single particle tracking and their remodeling by mutations and inhibition. Sci. Rep. 9, 16169 (2019).
Kæstel-Hansen, J. DeepSPT code. University of Copenhagen https://doi.org/10.17894/ucph.927973c8-6821-49c8-abae-6cd5996f1c47 (2024).
Kæstel-Hansen, J. DeepSPT code and models. University of Copenhagen https://doi.org/10.17894/ucph.25800387-29f5-4815-ae49-9a9d4d063bc4 (2024).
We thank members of our laboratories for help and encouragement. We thank S. C. Harrison for the fruitful discussions on the rotavirus data interpretation. We thank N. Kumar Mishra and K. J. Jensen for discussion concerning the preparation and use of insulin for our experiments. This work was funded by the Villum Foundation by being part of BioNEC (grant 18333) to J.K.-H. and N.S.H., the Novo Nordisk foundation challenge center for Optimised Oligo escape (NNF23OC0081287) (N.S.H. PI and T.K., co-PI), the center for 4D cellular dynamics (NNF22OC0075851). N.S.H. is affiliated with The Novo Nordisk Foundation Center for Protein Research funded by a generous donation from the Novo Nordisk Foundation (grant no. NNF14CC0001). J.K.-H., A.J.N., S.V.B. and N.S.H. are members of the Integrative Structural Biology Cluster at the University of Copenhagen. VILLUM FONDEN (40516) for W.B. and N.S.H. The Novo Nordisk Foundation Center for Basic Machine Learning Research in Life Science (NNF20OC0062606) for W.B. T.K. acknowledges support from NIH Maximizing Investigators’ Research Award GM130386, NIH grant AI163019, 1R01, IONIS Pharmaceuticals. M.d.S was supported by NIH/NCI CA13202 grant to S. C. Harrison.
The authors declare no competing interests.
Nature Methods thanks Koen Martens and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Rita Strack, in collaboration with the Nature Methods team. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Comparison of classification accuracy of DeepSPT, LSTM-based model29, and rolling MSD for various number of diffusion types. a, Confusion matrices of DeepSPT predictions per frame on 2D simulated test set trajectories (N=20000 tracks), Data displayed for either 4 diffusional states, or 3 states normal, directed and confined/subdiffusive, or 2 states normal/directed, confined/subdiffusive. b, Confusion matrices of attention BiLSTM29 predictions per frame on 2D simulated test set trajectories. Data for for 4, 3, 2 diffusional states as above c, Confusion matrices of rolling MSD12,47 predictions per frame on 2D simulated test set trajectories (predicts three classes as it bases predictions on the alpha exponent in a MSD fit based on (subdiffusive alpha)<(normal alpha)<(directed alpha)) Data for for 4, 3, 2 diffusional states as above. d, DeepSPT on 3D test set trajectories when combining classes into three and two respectively.
Investigation of model generalizability and limitations for key simulation parameters for traces containing all 4 diffusional types at random (see Methods for test set elaboration): a, Varying ranges of diffusion coefficients (D) for simulated trajectories (varying D is equivalent to varying temporal resolution for observation of a diffusing particle due to scale invariance of diffusion). b, step length to localization error ratio, i.e ratio of contribution to displacements from actual diffusion and localization error respectively where a value of 1 signifies an equal contribution and >1 signifies actual diffusion being larger. c, Duration of track. Median accuracy and mean accuracy are track-level metrics providing descriptive statistics for the distribution track-level accuracies for each test set track (N=20000). Flattened accuracy measures the accuracy for all frame-level predictions inside each trajectory pooled together.
Investigation of model generalizability and limitations for key simulation parameters for traces containing all 4 diffusional types at random (see Methods for test set elaboration): a, Varying ranges of diffusion coefficients (D) for simulated trajectories (varying D is equivalent to varying temporal resolution for observation of a diffusing particle due to scale invariance of diffusion). b, step length to localization error ratio, i.e ratio of contribution to displacements from actual diffusion and localization error respectively where a value of 1 signifies an equal contribution and >1 signifies actual diffusion being larger. c, Duration of track. Median accuracy and mean accuracy are track-level metrics providing descriptive statistics for the distribution track-level accuracies for each test set track (N=20000). Flattened accuracy measures the accuracy for all frame-level predictions inside each trajectory pooled together.
1000 trajectories are simulated within a box of varying dimensions and tracked using Trackpy56 (see Methods). The trajectories obtained by Trackpy are analyzed using DeepSPT’s temporal segmentation module. a, Example of trajectories produced by Trackpy showing tracks with one or more linking errors (red), correct/preserved tracks (black), and tracks without linking errors but still differ from the simulated tracks (grey). b, Example of simulated ground truth tracks (colored by ID). c, The median accuracy of DeepSPT in prediction diffusional behavior per time point per track for varying simulated box dimensions, thus varying degrees of linking errors. Median accuracy refers to the median of the distribution of correctly predicted time points in each individual track. Each blue cross represents an independent set of 1000 trajectories. Three sets are simulated per simulated box dimension with black dots and error bars representing the mean and standard deviations across each of the three sets. Red line represents the median accuracy of DeepSPT directly on simulated trajectories. d, Zoom-in, showing the dashed grey box in (c). e, The median accuracy of DeepSPT in prediction diffusional behavior per time point per track versus various degrees of preserved tracks, that is number of tracks perfectly tracked. As in (c) each blue cross represents an independent set of 1000 trajectories with three sets per simulated box dimension with black dots and error bars representing the mean and standard deviations across each of the three sets. Red line represents the median accuracy of DeepSPT directly on simulated trajectories. f, Zoom-in, showing the dashed grey box in (e).
Confusion matrix for all individual time point predictions within the 20000 2D (a, c, e) and 3D (b, d) test set trajectories simulated using the 2021 AnDi challenge task 3 open-source framework totalling 4 million predictions. See Muñoz-gil et al.28 for further test set specification. Diagonal entries are correct predictions and off-diagonal indicates confused classes. Each entry reports the percentage of predictions normalized to the actual number of true labels in the given class. a, Confusion matrix for DeepSPT on 2D trajectories. b, Confusion matrix for DeepSPT on 3D trajectories. c, Confusion matrix for Method E on 2D trajectories. d, Confusion matrix for Method E on 3D trajectories. e, Confusion matrix for Method J on 2D trajectories.
Confusion matrix for all individual time point predictions within the 20000 2D (a, c, e) and 3D (b, d) test set trajectories simulated by combining the anomalous diffusion behaviors from the 2021 AnDi challenge open-source framework28 into heterogeneous trajectories sampling multiple diffusional behaviors with multiple change points totalling 4 million predictions. Diagonal entries are correct predictions and off-diagonal indicates confused classes. Each entry reports the percentage of predictions normalized to the actual number of true labels in the given class. a, Confusion matrix for DeepSPT on 2D trajectories. b, Confusion matrix for DeepSPT on 3D trajectories. c, Confusion matrix for Method E on 2D trajectories. d, Confusion matrix for Method E on 3D trajectories. e, Confusion matrix for Method J on 2D trajectories.
a-e, Predicted changepoints (CP) versus true changepoints. Trajectories are constructed by combining two populations with overlapping diffusional properties into individual tracks with one changepoint (see Methods). Black line represents perfect classification. a, HMM-bayes17, 1556 data points compared to 5000 for other approaches due to computational time restraints as HMM-bayes requires several minutes per track. b, Rolling MSD. c, Original diffusional fingerprinting (Pinholt et al.13) d, Method E from the 2021 AnDi Challenge. e, DeepSPT. f, Table of classification metrics for the temporal segmentation of trajectory timepoints as post- and pre-uncoating and changepoint prediction. Median accuracy measures the median accuracy per trajectory (N=100). F1-score measures the F1-score of all individual timepoint predictions. Median frame error measures the median of absolute distances between predicted and true changepoints across all trajectories. Mean frame error measures the mean of absolute distances between predicted and true changepoints across all trajectories. Mean squared error measures the mean of the second norm distances between predicted and true changepoints across all trajectories. R2 measures the coefficient of determination.
a-e, Predicted changepoints (CP) versus true changepoints. Black line represents perfect classification. a, HMM-bayes17 b, Rolling MSD. c, Original diffusional fingerprinting (Pinholt et al.13) d, Method E from the 2021 AnDi Challenge. e, DeepSPT. f, Table of classification metrics for the temporal segmentation of trajectory timepoints as post- and pre-uncoating and changepoint prediction. Median accuracy measures the median accuracy per trajectory per time point (N=100), that is all time points before a changepoint is defined as predicted ‘before’, while time points after are defined as predicted ‘after’. These predictions are compared to the ground truth. F1-score measures the F1-score of all individual time point predictions as for median accuracy. Median frame error measures the median of absolute distances between predicted and true changepoints across all trajectories. Mean frame error measures the mean of absolute distances between predicted and true changepoints across all trajectories. Mean squared error measures the mean of the second norm distances between predicted and true changepoints across all trajectories. R2 measures the coefficient of determination.
Confusion matrices displaying classification performance on outputting NPC1 and EAA1 identity based solely on diffusion. a, DeepSPT. b, Original diffusional fingerprinting (Pinholt et al.13). c, Method E of the ANDI challenge. d, Utilizing the anomalous diffusion exponent (alpha) and diffusion coefficient. e, Only variations in diffusion coefficient. f, Only variations in alpha.
Confusion matrices displaying classification performance of outputting whether AP2 is dorsal or ventral based solely on diffusion. a, DeepSPT. b, Original diffusional fingerprinting (Pinholt et al.13). c, Method E of the ANDI challenge. d, Utilizing the anomalous diffusion exponent (alpha) and diffusion coefficient. e, Only variations in diffusion coefficient. f, Only variations in alpha.
Supplementary Figs. 1–26, materials and Tables 1–4.
Single rotavirus tracking in 3D by LLSM observing viral uncoating along with DeepSPT identification of the event in split channel view.
Single rotavirus tracking in 3D by LLSM observing viral uncoating along with DeepSPT identification of the event in merged channel view.
Single-particle trajectories of NPC1-positive endosomes in 3D in live cell by LLSM.
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Kæstel-Hansen, J., de Sautu, M., Saminathan, A. et al. Deep learning-assisted analysis of single-particle tracking for automated correlation between diffusion and function. Nat Methods (2025). https://doi.org/10.1038/s41592-025-02665-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41592-025-02665-8