Breast cancer mitotic cell detection using cascade convolutional neural network with U-Net

Xi Lu; Zejun You; Miaomiao Sun; Jing Wu; Zhihong Zhang; Xi Lu; Zejun You; Miaomiao Sun; Jing Wu; Zhihong Zhang

doi:10.3934/mbe.2021036

Mathematical Biosciences and Engineering

2021, Volume 18, Issue 1: 673-695. doi: 10.3934/mbe.2021036

Previous Article Next Article

Research article

Breast cancer mitotic cell detection using cascade convolutional neural network with U-Net

1.
School of Mechanical Engineering, Southeast University, Nanjing 211189, China
2.
Department of Pathology, The First Affiliated Hospital of Nanjing Medical University, Nanjing 210029, China

Received: 23 September 2020 Accepted: 15 December 2020 Published: 18 December 2020

The number of mitotic tumor cells detected in each slide is one of the key indicators of breast cancer prognosis. However, accurate mitotic cell counts are still a difficult problem for pathologists and related experts. Traditional methods use manual design algorithms to extract features of mitotic cells, and most methods rely on sliding windows to achieve pixel-level classification through deep learning. However, the complex background and high resolution of pathological images make the above methods time-consuming and ineffective. In order to solve the above problems, we propose a new cascaded convolutional neural network UBCNN (cascaded CNN based on UNet), which consists of three parts: semantic segmentation and classification to detect mitosis. First, we use an improved UNet ++ segmentation network to locate the candidate set of mitotic targets. Secondly, an adequately labeled cell nucleus data set is sent to an improved two-dimensional VNet network, and the cell nucleus is located by means of semantic segmentation to obtain accurate image blocks of mitotic and non-mitotic cells. Finally, the obtained cell image block is used to train a convolutional neural network to achieve binary classification, and the candidate set area is screened to retain the final result of mitosis cells. This paper verifies the detection effect of the above-mentioned cascade detection algorithm on the ICPR 2012 and 2014 mitosis automatic detection competition data sets. The evaluation indicators include accuracy, recall and F-score. Our cascade detection algorithm based on segmentation and classification reached 0.831 on the ICPR 2012 data set and 0.576 on the ICPR 2014 data set. Compared with other existing algorithms, the detection effect was improved, which was very competitive.
- mitosis automatic detection,
- breast cancer, deep learning,
- cascade detection,
- semantic segmentation,
- binary classification
Citation: Xi Lu, Zejun You, Miaomiao Sun, Jing Wu, Zhihong Zhang. Breast cancer mitotic cell detection using cascade convolutional neural network with U-Net[J]. Mathematical Biosciences and Engineering, 2021, 18(1): 673-695. doi: 10.3934/mbe.2021036

Related Papers:

Abstract

The number of mitotic tumor cells detected in each slide is one of the key indicators of breast cancer prognosis. However, accurate mitotic cell counts are still a difficult problem for pathologists and related experts. Traditional methods use manual design algorithms to extract features of mitotic cells, and most methods rely on sliding windows to achieve pixel-level classification through deep learning. However, the complex background and high resolution of pathological images make the above methods time-consuming and ineffective. In order to solve the above problems, we propose a new cascaded convolutional neural network UBCNN (cascaded CNN based on UNet), which consists of three parts: semantic segmentation and classification to detect mitosis. First, we use an improved UNet ++ segmentation network to locate the candidate set of mitotic targets. Secondly, an adequately labeled cell nucleus data set is sent to an improved two-dimensional VNet network, and the cell nucleus is located by means of semantic segmentation to obtain accurate image blocks of mitotic and non-mitotic cells. Finally, the obtained cell image block is used to train a convolutional neural network to achieve binary classification, and the candidate set area is screened to retain the final result of mitosis cells. This paper verifies the detection effect of the above-mentioned cascade detection algorithm on the ICPR 2012 and 2014 mitosis automatic detection competition data sets. The evaluation indicators include accuracy, recall and F-score. Our cascade detection algorithm based on segmentation and classification reached 0.831 on the ICPR 2012 data set and 0.576 on the ICPR 2014 data set. Compared with other existing algorithms, the detection effect was improved, which was very competitive.

References

[1]	C. W. Elston, I. O. Ellis, Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: Experience from a large study with long-term follow-up, Histopathology, 19 (1991), 403–410. doi: 10.1111/j.1365-2559.1991.tb00229.x
[2]	G. Jiménez, D. Racoceanu, Deep learning for semantic segmentation vs. classification in computational pathology: Application to mitosis analysis in breast cancer grading, Front. Bioeng. Biotechnol., (2019), 7–145.
[3]	Mitosis Detection Contest[EB/OL], http://ipal.cnrs.fr/ICPR2012/.
[4]	MITOS-ATYPIA-14 Contest[EB/OL], https://mitos-atypia-14.grand-challenge.org/home/.
[5]	C. Sommer, L. Fiaschi, F. A. Hamprecht, D. W. Gerlich, Learning-based mitotic cell detection in histopathological images, Proc. 21st Int. Conf. Pattern Recognit., (2012), 2306–2309.
[6]	S. Berg, D. Kutra, T. Kroeger, C. N. Straehle, B. X. Kausler, C. Haubold, et al., Ilastik: Interactive machine learning for (bio)image analysis, Nat. Methods, (2019), 1226–1232.
[7]	C. H. Huang, H. K. Lee, Automated mitosis detection based on exclusive independent component analysis, Proc. 21st Int. Conf. Pattern Recognit., (2012), 1856–1859.
[8]	A. M. Khan, H. El-Daly, N. M. Rajpoot, A gamma-gaussian mixture model for detection of mitotic cells in breast cancer histopathology images, Proc. 21st Int. Conf. Pattern Recognit., (2012), 149–152.
[9]	H. Irshad, Automated mitosis detection in histopathology using morphological and multi-channel statistics features, J. Pathol. Inf., 4 (2013), 10.
[10]	A. Tashk, M. S. Helfroush, H. Danyali, M. Akbarzadeh-jahromi, Automatic detection of breast cancer mitotic cells based on the combination of textural, statistical and innovative mathematical features, Appl. Math. Modell., 39 (2015), 6165–6182.
[11]	F. Pourakpour, H. Ghassemian, Automated mitosis detection based on combination of effective textural and morphological features from breast cancer histology slide images, Biomed. Eng. IEEE, 39 (2016), 214–223.
[12]	H. Wang, A. Cruzroa, A. Basavanhally, H. Gilmore, N. Shih, M. Feldman, et al., Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features, J. Med. Imag., 1 (2014), 034003.
[13]	T. Kausar, M. Wang, B. Wu, M. Idrees, B. Kanwal, Multi-scale deep neural network for mitosis detection in histological images, Proc. Int. Conf. Intell. Informat. Biomed. Sci., (2018), 47–51.
[14]	D. C. Cireşan, A. Giusti, L. M. Gambardella, J. Schmidhuber, Mitosis detection in breast cancer histology images with deep neural networks, Int. Conf Med. Image Comput. Comput. Assisted Intervention, (2013), 411–418.
[15]	H. Chen, Q. Dou, X. Wang, J. Qin, P. A. Heng, Mitosis detection in breast cancer histology images via deep cascaded networks, Proc. 30th AAAI Conf. Artif. Intell., (2016), 1160–1166.
[16]	M. Ma, Y. Shi, W. Li, Y. Gao, J. Xu, A novel two-stage deep method for mitosis detection in breast cancer histology images, Proc. 24th Int. Conf. Pattern Recognit., (2018), 3892–3897.
[17]	N. Maroof, A. Khan, S. A. Qureshi, A. Rehman, R. K. Khalil, S. Shim, Mitosis detection in breast cancer histopathology images using hybrid feature space, Photodiagn. Photodyn. Ther., (2020), 101885.
[18]	Y. Zhang, J. Chen, X. Pan, Multi-feature fusion of deep networks for mitosis segmentation in histological images, Int. J. Imag. Syst. Technol., (2020), 1–13.
[19]	M. Hwang, D. Wang, C. Wu, W. C. Jiang, X. X. Kong, K. Hwang, et al., A fuzzy segmentation method to learn classification of mitosis, Int. J. Fuzzy Syst., (2020), 1653–1664.
[20]	C. Li, X. Wang, W. Liu, L. J. Latecki, DeepMitosis: Mitosis detection via deep detection, verification and segmentation networks, Med. Image Anal., (2018), 121–133.
[21]	S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. (2017), 1137–1149.
[22]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, Proceed. IEEE Conf. Comput. Vision Pattern Recognit., (2016), 770–778.
[23]	H. Lei, S. Liu, H. Xie, J. Y. Kuo, B. Lei, An improved object detection method for mitosis detection, Proc. 41st Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., (2019), 130–133.
[24]	A. Sengupta, Y. Ye, R. Wang, C. Liu, K. Roy, Going deeper in spiking neural networks: VGG and residual architectures, Front. Neurosci., (2019), 1–10
[25]	C. Li, X. Wang, W. Liu, L. J. Latecki, B. Wang, J. Huang, Weakly supervised mitosis detection in breast histopathology images using concentric loss, Med. Image Anal. (2019), 165–178.
[26]	K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask R-CNN, Proceed. IEEE Int. Conf. Comput. Vision, (2017), 2980–2988.
[27]	V. Dodballapur, Y. Song, H. Huang, M. Chen, W. Chrzanowski, W. Cai, Mask-driven mitosis detection in histopathology images, Proceed. IEEE 16th Int. Symp. Biomed. Imaging, (2019), 1855–1859.
[28]	D. Cai, X. Sun, N. Zhou, X. Han, J. Yao, Efficient mitosis detection in breast cancer histology images by RCNN, Proceed. IEEE 16th Int. Symp. Biomed. Imaging, (2019), 919–922.
[29]	Y. Li, E. Mercan, S. Knezevitch, J. G. Elmore, L. G. Shapiro, Efficient and accurate mitosis detection, a lightweight RCNN approach, Proceed. 7th Int. Conf Pattern Recognit. Appl. Methods, (2018), 69–77.
[30]	H. J. Lei, S. M. Liu, A. Elazab, X. H. Gong, B. Y. Lei, Attention-guided multi-branch convolutional neural network for mitosis detection from histopathological images, J. Biomed. Health Inf., (2020), 2168–2194.
[31]	T. Mahmood, M. Arsalan, M. Owais, M. B. Lee, K. R. Park, Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster R-CNN and deep CNNs, J. Clin. Med., (2020), 749
[32]	S. Meriem, T. Wang, S. A. Al-Fadhli, PartMitosis: A partially supervised deep learning framework for mitosis detection in breast cancer histopathology images, IEEE Access, (2020), 1–1.
[33]	S. Meriem, X. Wang, T. Wang, MaskMitosis: A deep learning framework for fully supervised, weakly supervised, and unsupervised mitosis detection in histopathology images, Med. Biol. Eng. Comput., (2020) 1603–1623.
[34]	N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane, A. Sethi, A dataset and a technique for generalized nuclear segmentation for computational pathology, IEEE Trans. Med. Imaging, 36 (2017), 1550–1560. doi: 10.1109/TMI.2017.2677499
[35]	O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, Int. Conf Med. Image Comput. Comput. Assisted Intervention, (2015), 234–241.
[36]	Z. W. Zhou, M. Mahfuzur, R. Siddiquee, N Tajbakhsh, J. M. Liang, UNet ++: A nested U-Net architecture for medical image segmentation, Dlmia, (2018), 1–7.
[37]	C. Y. Lee, S. Xie, P. Gallagher, Z. Zhang, Z. W. Tu, Deeply-supervised nets, Proceed. 2015 18th Int. Conf. Artif. Int. Stat., (2015), 562–570.
[38]	C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, M. J. Cardoso, Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations, Comput. Vision Pattern Recogn., 2017.
[39]	C. Yamauchi, T. Hasebe, M. Iwasaki, S. Imoto, N. Wada, M. Fukayama, et al., Accurate assessment of lymph vessel tumor emboli in invasive ductal carcinoma of the breast according to tumor areas, and their prognostic significance, Hum. Pathol., 38 (2007), 247–259.
[40]	H. Wang, A. Cruz-Roa, A. Basavanhally, H. Gilmore, N. Shih, M. Feldman, et al., Cascaded ensemble of convolutional neural networks and handcrafted features for mitosis detection, Proc. SPIE, 9041 (2014).
[41]	F. Milletari, N. Navab, S. Ahmadi, V-Net: Fully convolutional neural networks for volumetric medical image segmentation, proc. Int. Conf. 3D Vision, (2016), 565–571.
[42]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, Proc. IEEE Conf. Comput. Vis. Pattern Recognit., (2016), 770–778.
[43]	P. Vamplew, R. Dazele, C. Foale, Softmax exploration strategies for multi-objective reinforcement learning, Neuro comput., (2017), 74–86.
[44]	A. Vahadane, T. Peng, S. Albarqouni, M. Baust, K. Steiger, Structure-preserved color normalization for histological images, IEEE Int. Symp. Biomed. Imaging, (2015), 1012–1015.
[45]	A. Krizhevsky, I. Sutskever, G. E. Hinton, ImageNet classification with deep convolutional neural networks, Int. Conf. Neural Inf. Process. Syst., (2012), 1097–1105.
[46]	N. Otsu, A threshold selection method from gray-level histograms, IEEE Trans. Syst. Man. Cybern., 9 (1979), 62–66. doi: 10.1109/TSMC.1979.4310076
[47]	M. Veta, J. Pluim, Detecting mitotic figures in breast cancer histopathology images, SPIE Med. Imaging, 8676 (2013), 867607.
[48]	C. D. Malon, C. Eric, Classification of mitotic figures with convolutional neural networks and seeded blob features, J. Pathol. Inf., 4 (2013), 9.
[49]	M Veta, P. J. Diest, S. M. Willems, H. Wang, A. Madabhushi, A. Cruz-Roa, et al., Assessment of algorithms for mitosis detection in breast cancer histopathology images, Med. Image Anal., 20 (2015), 237–248.
[50]	H. Chen, X. Wang, P. A. Heng, Automated mitosis detection with deep regression networks, 2016 IEEE 13th Int. Symp. Biomed. Imaging, 2016, 1945–8452.
[51]	C. Li, X. Wang, W. Liu, L. J. Latecki, DeepMitosis: Mitosis detection via deep detection, verification and segmentation networks, Med. Image Anal., 45 (2018), 121–133.

Reader Comments

Your name:*

Email:*
© 2021 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)