Abstract
Deep learning is now a powerful tool in microscopy data analysis, and is routinely used for image processing applications such as segmentation and denoising. However, it has rarely been used to directly learn mechanistic models of a biological system, owing to the complexity of the internal representations. Here, we develop an end-to-end machine learning approach capable of learning an explainable model of a complex biological phenomenon, cell competition, directly from a large corpus of time-lapse microscopy data. Cell competition is a quality control mechanism that eliminates unfit cells from a tissue, during which cell fate is thought to be determined by the local cellular neighbourhood over time. To investigate this, we developed a new approach (τ-VAE) by coupling a probabilistic encoder to a temporal convolution network to predict the fate of each cell in an epithelium. Using the τ-VAE’s latent representation of the local tissue organization and the flow of information in the network, we decode the physical parameters responsible for correct prediction of fate in cell competition. Remarkably, the model autonomously learns that cell density is the single most important factor in predicting cell fate—a conclusion that is in agreement with our current understanding from over a decade of scientific research. Finally, to test the learned internal representation, we challenge the network with experiments performed in the presence of drugs that block signalling pathways involved in competition. We present a novel discriminator network, which using the predictions of the τ-VAE can identify conditions that deviate from the normal behaviour, paving the way for automated, mechanism-aware drug screening.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Training data including cell images, single-cell trajectories and metadata are available from the UCL data repository37. For other enquiries contact the corresponding author. Source data are provided with this paper.
Code availability
A reference implementation of the τ-VAE is available at https://github.com/lowe-lab-ucl/cellx-predict and the UCL software repository38. For other enquiries contact the corresponding author.
References
Levayer, R. & Moreno, E. Mechanisms of cell competition: themes and variations. J. Cell Biol. 200, 689–698 (2013).
Morata, G. & Ripoll, P. Minutes: mutants of Drosophila autonomously affecting cell division rate. Dev. Biol. 42, 211–221 (1975).
Parker, T., Madan, E., Gupta, K., Moreno, E. & Gogna, R. Cell competition spurs selection of aggressive cancer cells. Trends Cancer 6, 732–736 (2020).
Levayer, R., Hauert, B. & Moreno, E. Cell mixing induced by myc is required for competitive tissue invasion and destruction. Nature 524, 476–480 (2015).
Vincent, J.-P., Fletcher, A. G. & Baena-Lopez, L. A. Mechanisms and mechanics of cell competition in epithelia. Nat. Rev. Mol. Cell Biol. 14, 581–591 (2013).
Hogan, C. et al. Characterization of the interface between normal and transformed epithelial cells. Nat. Cell Biol. 11, 460–467 (2009).
Wagstaff, L. et al. Mechanical cell competition kills cells via induction of lethal p53 levels. Nat. Commun. 7, 11373 (2016).
Bove, A. et al. Local cellular neighborhood controls proliferation in cell competition. Mol. Biol. Cell 28, 3215–3228 (2017).
Gradeci, D. et al. Cell-scale biophysical determinants of cell competition in epithelia. eLife 10, e61011 (2021).
Moen, E. et al. Deep learning for cellular image analysis. Nat. Methods 16, 1233–1246 (2019).
Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).
Ren, E. et al. Deep learning-enhanced morphological profiling predicts cell fate dynamics in real-time in hPSCs. Preprint at bioRxiv https://doi.org/10.1101/2021.07.31.454574 (2021).
Kingma, D. P. & Welling, M. Auto-encoding variational bayes. Preprint at http://arxiv.org/abs/1312.6114 (2013).
Chan, C. K., Hadjitheodorou, A., Tsai, T. Y.-C. & Theriot, J. A. Quantitative comparison of principal component analysis and unsupervised deep learning using variational autoencoders for shape analysis of motile cells. Preprint at bioRxiv https://doi.org/10.1101/2020.06.26.174474 (2020).
Zaritsky, A. et al. Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. Cell Syst. 12, 733–747.e6 (2021).
Wu, Z. et al. DynaMorph: self-supervised learning of morphodynamic states of live cells. Mol. Biol. Cell 33, e21110561 (2022).
Yang, K. D. et al. Predicting cell lineages using autoencoders and optimal transport. PLoS Comput. Biol. 16, e1007828 (2020).
Buggenthin, F. et al. Prospective identification of hematopoietic lineage choice by deep learning. Nat. Methods 14, 403–406 (2017).
Norman, M. et al. Loss of Scribble causes cell competition in mammalian cells. J. Cell Sci. 125, 59–66 (2012).
Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. Lect. Notes Comput. Sci. 9351, 234–241 (2015).
Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Front. Comput. Sci. 3 (2021).
Mnih, V., Heess, N., Graves, A. & Kavukcuoglu, K. Recurrent models of visual attention. In Advances in Neural Information Processing Systems Vol. 27, 2204-2212 (eds Ghahramani, Z. et al.) (Curran Associates, 2014); https://proceedings.neurips.cc/paper/2014/file/09c6c3783b4a70054da74f2538ed47c6-Paper.pdf
Higgins, I. et al. beta-VAE: learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proc. (2017); https://openreview.net/forum?id=Sy2fzU9gl
van den Oord, A. et al. Wavenet: a generative model for raw audio. Preprint at http://arxiv.org/abs/1609.03499 (2016).
Bai, S., Kolter, J. Z. & Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. Preprint at http://arxiv.org/abs/1803.01271 (2018).
Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).
Levayer, R., Dupont, C. & Moreno, E. Tissue crowding induces caspase-dependent competition for space. Curr. Biol. 26, 670–677 (2016).
Smilkov, D., Thorat, N., Kim, B., Viégas, F. B. & Wattenberg, M. Smoothgrad: removing noise by adding noise. Preprint at http://arxiv.org/abs/1706.03825 (2017).
Kuma, Y. et al. BIRB796 inhibits all p38 MAPK isoforms in vitro and in vivo. J. Biol. Chem. 280, 19472–19479 (2005).
Chandrasekaran, S. N., Ceulemans, H., Boyd, J. D. & Carpenter, A. E. Image-based profiling for drug discovery: due for a machine-learning upgrade? Nat. Rev. Drug Discov. 20, 145–159 (2020).
Kucinski, I., Dinan, M., Kolahgar, G., & Piddini, E. Chronic activation of JNK JAK/STAT and oxidative stress signalling causes the loser cell status. Nat. Commun. 8, 136 (2017).
Parmar, N. et al. Image transformer. Preprint at http://arxiv.org/abs/1802.05751 (2018).
Abnar, S. & Zuidema, W. H. Quantifying attention flow in transformers. Preprint at https://arxiv.org/abs/2005.00928 (2020).
Hetzel, L., Fischer, D. S., Günnemann, S. & Theis, F. J. Graph representation learning for single-cell biology. Curr. Opin. Syst. Biol. 28, 100347 (2021).
Burgess, C. P. et al. Understanding disentangling in β-VAE. Preprint at https://arxiv.org/abs/1804.03599 (2018).
Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. Preprint at https://arxiv.org/abs/1411.4038 (2015).
Lowe, A. R., Soelistyo, C. J., Vallardi, G. & Charras, G. cellX-predict datasets. UCL Research Data Repository https://doi.org/10.5522/04/16578959 (2022).
Lowe, A. R. & Soelistyo, C. J. cellX-predict software. UCL Software Database https://doi.org/10.5522/04/19207923 (2022).
Acknowledgements
This work was supported by a BBSRC LIDo artificial intelligence PhD studentship for C.J.S. G.V. was supported by BBSRC grant BB/S009329/1. We thank N. Day, J. Michalowska and D. Smaje for help with annotating data, and M. Kelkar for additional supporting data. We thank Y. Fujita for the kind gift of cell lines used in this work. We also thank members of the Lowe and Charras laboratories for discussions and technical support during the project. A.R.L. acknowledges the Turing Fellowship from the Alan Turing Institute. A.R.L. and G.C. acknowledge the support of BBSRC grant BB/S009329/1.
Author information
Authors and Affiliations
Contributions
A.R.L. and G.C. conceived and designed the research. G.V. performed experiments. C.J.S. developed and performed computational analysis. A.R.L. wrote the image processing and cell tracking code. C.J.S., G.V., G.C. and A.R.L. evaluated the results and wrote the paper.
Corresponding author
Ethics declarations
Competing interests
A UK provisional patent application (patent application no. GB2116864.6) filed in relation to these results by applicant UCL Business Ltd remains pending. The remaining authors declare no competing interests.
Peer review
Peer review information
Nature Machine Intelligence thanks Shalin Mehta and Chris Bakal for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Data flow in the model.
(a) Example time-lapse microscopy data showing a mixed population of MDCKWT (green) and scribkd(magenta) cells. (b) Single-cell tracking is used to build a detailed training dataset of trajectories. The single-cell track is used to extract a glimpse of the cell over time, that becomes the input data for the machine learning models. (c) The data preparation and inference pipeline. A CNN/LSTM network classifies the fate of the cell and determines the cutoff point to truncate the track to remove images that encode the fate of the cell. The goal of the machine learning model is then to learn a representation that can predict the fate of a cell (circled in white) given the local configuration during interphase. Importantly, the model does not actually observe the fate since these data fall beyond the cutoff. Images are taken at 4-minute intervals, MDCKWT cells appear in green and scribkdin magenta.
Extended Data Fig. 2 Glimpse extraction and cell masking to determine the best image input scale for prediction.
Three different scale windows are extracted, Small, Mid and Large, corresponding to 21 × 21 μm, 42 × 42 μm and 84 × 84 μm FOV respectively. For the mid-scale, we also perform masking, by removing either the neighbor cells or the central cell to determine the important features for prediction.
Extended Data Fig. 3 Generative modeling of ‘synthetic’ trajectories.
For each synthetic trajectory we start by encoding a real image as a starting point. Next, we take a random walk in the latent space. These trajectories in latent space are used as inputs to the TCN. Here, we also use the decoder to generate image sequences that represent the random walks in latent space.
Supplementary information
Supplementary Information
Supplementary Methods, Figs. 1–19 and Tables 1–3.
Supplementary Video 1
Time-lapse acquisition and tracking of single cell, showing three different spatial scales extracted to form the glimpse.
Supplementary Video 2
Glimpse extracted from Supplementary Video 1.
Supplementary Video 3
Example cell detection and tracking for MDCKWT:scribkd dataset.
Supplementary Video 4
Example cell detection and tracking for MDCKWT:scribkd,tet− dataset.
Supplementary Video 5
Example cell detection and tracking for MDCKWT:scribkd + 2 μM BIRB 796 dataset.
Supplementary Video 6
Example τ -VAE output for MDCKWT:scribkd dataset.
Supplementary Video 7
Example τ -VAE output for MDCKWT:scribkd,tet− dataset.
Supplementary Video 8
Example τ -VAE output for MDCKWT:scribkd + 2 μM BIRB 796 dataset.
Source data
Source Data Fig. 2
Statistical source data.
Source Data Fig. 3
Statistical source data.
Source Data Fig. 4
Statistical source data.
Rights and permissions
About this article
Cite this article
Soelistyo, C.J., Vallardi, G., Charras, G. et al. Learning biophysical determinants of cell fate with deep neural networks. Nat Mach Intell 4, 636–644 (2022). https://doi.org/10.1038/s42256-022-00503-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s42256-022-00503-6
This article is cited by
-
Detecting abnormal cell behaviors from dry mass time series
Scientific Reports (2024)
-
Bridging live-cell imaging and next-generation cancer treatment
Nature Reviews Cancer (2023)
