Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

High-accuracy prostate cancer pathology using deep learning

Nature Machine Intelligence volume 2, pages 411–418 (2020)Cite this article

Subjects

Abstract

Deep learning (DL) is a powerful methodology for the recognition and classification of tissue structures in digital pathology. Its performance in prostate cancer pathology is still under intensive investigation. Here we develop DL-based models for the detection of prostate cancer tissue in whole-slide images based on a large high-quality annotated training dataset and a modern state-of-the-art convolutional network architecture (NASNetLarge). The overall accuracy of our model for tumour detection in two validation cohorts is comparable to that of pathologists and reaches 97.3% in a native version and more than 98% using the suggested DL-based augmentation strategies. As a second step, we suggest a new biologically meaningful DL-based algorithm for Gleason grading of prostatic adenocarcinomas with high, human-level performance in prognostic stratification of patients when tested in several well-characterized validation cohorts. Furthermore, we determine the optimal minimal tumour size (real size of approximately 560 × 560 µm) for robust Gleason grading representative of the whole tumour focus. Our approach is realized in the unified digital pathology pipeline, which delivers all the relevant tumour metrics for a pathology report.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Composition of training and validation cohorts.
Fig. 2: Analysis of the model accuracy.
Fig. 3: Pipeline for analysis of prostatectomy WSIs.
Fig. 4: Tests of grading accuracy for three pathologists and DL-based model.
Fig. 5: A grading part of pipeline for processing of WSIs.

Similar content being viewed by others

Data availability

The whole-slide images used for algorithm development (training dataset) are publicly available through GDC Data Portal of the National Cancer Institute (The Cancer Genome Atlas Project; http://portal.gdc.cancer.gov). The validation datasets (image patches of tumour and benign classes) generated and analysed during the current study are available for academical use only from public repository https://zenodo.org/deposit/3825933. Any usage for publications should be consented by corresponding authors. All other data may be obtained upon request to the authors.

Code availability

The source code used in this study is available at https://github.com/gagarin37/deep_learning_pca.

References

  1. Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. & Madabhushi, A. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019).

    Article  Google Scholar 

  2. Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

    Article  Google Scholar 

  3. Kather, J. N. et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 16, e1002730 (2019).

    Article  Google Scholar 

  4. Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25, 1054–1056 (2019).

    Article  Google Scholar 

  5. Bychkov, D. et al. Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep. 8, 3395 (2018).

    Article  Google Scholar 

  6. Couture, H. D. et al. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. npj Breast Cancer 4, 30 (2018).

    Article  Google Scholar 

  7. Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).

    Article  Google Scholar 

  8. Ehteshami Bejnordi, B. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199–2210 (2017).

    Article  Google Scholar 

  9. Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    Article  Google Scholar 

  10. Nagpal, K. et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. npj Digit. Med. 2, 48 (2019).

    Article  Google Scholar 

  11. Bulten, W. et al. Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard. Sci. Rep. 9, 864 (2019).

    Article  Google Scholar 

  12. Chen, C. M., Huang, Y. S., Fang, P. W., Liang, C. W. & Chang, R. F. A computer-aided diagnosis system for differentiation and delineation of malignant regions on whole-slide prostate histopathology image using spatial statistics and multidimensional DenseNet. Med. Phys. 47, 1021–1033 (2019).

    Article  Google Scholar 

  13. Poojitha, U. P. & Lal Sharma, S. Hybrid unified deep learning network for highly precise Gleason grading of prostate cancer. In Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 899–903 (IEEE, 2019).

  14. Li, J. et al. A multi-scale U-Net for semantic segmentation of histological images from radical prostatectomies. AMIA Annu. Symp. Proc. 2017, 1140–1148 (2017).

    Google Scholar 

  15. Li, J. et al. An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies. Comput. Med. Imaging Graph. 69, 125–133 (2018).

    Article  Google Scholar 

  16. Li, W. et al. Path R-CNN for prostate cancer diagnosis and gleason grading of histological images. IEEE Trans. Med. Imaging 38, 945–954 (2019).

    Article  Google Scholar 

  17. Lucas, M. et al. Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies. Virchows Arch. 475, 77–83 (2019).

    Article  Google Scholar 

  18. Ren, J. et al. Recurrence analysis on prostate cancer patients with Gleason score 7 using integrated histopathology whole-slide images and genomic data through deep neural networks. J. Med. Imaging 5, 047501 (2018).

    Article  Google Scholar 

  19. Arvaniti, E. et al. Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci. Rep. 8, 12054 (2018).

    Article  Google Scholar 

  20. Singh, M. et al. Gland segmentation in prostate histopathological images. J. Med. Imaging 4, 027501 (2017).

    Article  Google Scholar 

  21. Jiménez del Toro, O. et al. Convolutional neural networks for an automatic classification of prostate tissue slides with high-grade Gleason score. Proc. SPIE 10140, 101400O (2017).

  22. Bulten, W. et al. Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21, 233–241 (2020).

    Article  Google Scholar 

  23. Ström, P. et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 21, 222–232 (2020).

    Article  Google Scholar 

  24. Komura, D. & Ishikawa, S. Machine learning methods for histopathological image analysis. Comput. Struct. Biotechnol. J. 16, 34–42 (2018).

    Article  Google Scholar 

  25. Gertych, A. et al. Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides. Sci. Rep. 9, 1483 (2019).

    Article  Google Scholar 

  26. Nir, G. et al. Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med. Image Anal. 50, 167–180 (2018).

    Article  Google Scholar 

  27. Nir, G. et al. Comparison of artificial intelligence techniques to evaluate performance of a classifier for automatic grading of prostate cancer from digitized histopathologic images. JAMA Netw. Open 2, e190442 (2019).

    Article  Google Scholar 

  28. Li, J. et al. An attention-based multi-resolution model for prostate whole slide imageclassification and localization. Preprint at https://arxiv.org/abs/1905.13208 (2019).

  29. Gertych, A. et al. Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Comput. Med. Imaging Graph. 46 (Pt 2), 197–208 (2015).

    Article  Google Scholar 

  30. Litjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016).

    Article  Google Scholar 

  31. Zoph, B., Vasudevan, V., Shlens, J. & Le, Q. V. Learning transferable architectures for scalable image recognition. Preprinit at https://arxiv.org/abs/1707.07012 (2017).

  32. Tolkach, Y., Thomann, S. & Kristiansen, G. Three-dimensional reconstruction of prostate cancer architecture with serial immunohistochemical sections: hallmarks of tumour growth, tumour compartmentalisation, and implications for grading and heterogeneity. Histopathology 72, 1051–1059 (2018).

    Article  Google Scholar 

  33. van Royen, M. E. et al. Three-dimensional microscopic analysis of clinical prostate specimens. Histopathology 69, 985–992 (2016).

    Article  Google Scholar 

  34. Macenko, M. et al. A method for normalizing histology slides for quantitative analysis. In IEEE International Symposium on Biomedical Imaging: From Nano to Macro 1107–1110 (IEEE, 2009).

  35. Tolkach, Y. & Kristiansen, G. Cribriform and glomeruloid acinar adenocarcinoma of the prostate—an intratumoural intraductal carcinoma? Histopathology 74, 804–808 (2019).

    Article  Google Scholar 

  36. Humphrey, P. A., Moch, H., Cubilla, A. L., Ulbright, T. M. & Reuter, V. E. The 2016 WHO classification of tumours of the urinary System and male genital organs—part B: prostate and bladder tumours. Eur. Urol. 70, 106–119 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

The GPU card for this study was donated by the NVIDIA corporation (a GPU academic grant programme). No funding was received for this study.

Author information

Authors and Affiliations

  1. Institute of Pathology, University Hospital Cologne, Cologne, Germany

    Yuri Tolkach

  2. Institute of Pathology, University Hospital Bonn, Bonn, Germany

    Yuri Tolkach, Tilmann Dohmgörgen, Marieta Toma & Glen Kristiansen

Authors
  1. Yuri Tolkach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tilmann Dohmgörgen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marieta Toma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Glen Kristiansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceiving the study and the design: Y.T. Annotations, creating the datasets, developing and training the models, development of the grading algorithm and augmentation strategies, digital pathology pipeline, validation tests, hardware, conducting the statistical analysis and data analysis: Y.T. Providing validation datasets: G.K. Grading for validation experiments: Y.T., G.K. and M.T. Data interpretation: Y.T. and G.K. Drafting the manuscript: Y.T. Critically revising for important intellectual content: Y.T., T.D., M.T. and G.K.

Corresponding authors

Correspondence to Yuri Tolkach or Glen Kristiansen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Summary of study design and setup.

Study design and setup of single study steps, principles of patch generation (including number of patches generated in training and validation datasets in thousands of unique non-intersecting patches, k), approaches to training and validation. For details to certain steps see Methods.

Extended Data Fig. 2 Training principles of the model.

Three classes were used for training (benign glandular, benign non-glandular, tumour) with stain normalization. NASNetLarge architecture was used with addition of three new layers for classification (flatten, fully connected, and classification layers). Probability of being benign could be summarized from output probabilities of two benign classes. Training principles and parameters are presented in boxes.

Extended Data Fig. 3 Deep learning – based prostate cancer grading.

a. Prostate cancer (PCA) architecture is not just a rough mix of glands with Gleason patterns (GP) 3, 4, and 5. In three dimensions, carcinoma is a tree, where single GPs do not exist, being a continuum. b. Our Gleason grading algorithm is based on the understanding of GPs being a continuum. During processing, a tumour region is being cut into patches that are then separately analyzed by convnet (trained on pure GPs, see Materials and Methods). Probabilities of different classes (GP3, GP4, GP5) are counted for every single patch and summarized for the whole tumour region and not used for nomination of every patch to a discrete GP (as in classical paradigm).

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tolkach, Y., Dohmgörgen, T., Toma, M. et al. High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell 2, 411–418 (2020). https://doi.org/10.1038/s42256-020-0200-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-020-0200-7

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing