Modified UNet++ with atrous spatial pyramid pooling for blood cell image segmentation

Kun Lan; Jianzhen Cheng; Jinyun Jiang; Xiaoliang Jiang; Qile Zhang; Kun Lan; Jianzhen Cheng; Jinyun Jiang; Xiaoliang Jiang; Qile Zhang

doi:10.3934/mbe.2023064

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 1: 1420-1433. doi: 10.3934/mbe.2023064

Previous Article Next Article

Research article Special Issues

Modified UNet++ with atrous spatial pyramid pooling for blood cell image segmentation

1.
College of Mechanical Engineering, Quzhou University, Quzhou 324000, China
2.
Department of Rehabilitation, Quzhou Third Hospital, Quzhou 324000, China
3.
Department of Rehabilitation, The Quzhou Affiliated Hospital of Wenzhou Medical University, Quzhou People's Hospital, Quzhou 324000, China

† The authors contributed equally to this work.
Academic Editor: Simon James Fong

Received: 07 September 2022 Revised: 12 October 2022 Accepted: 21 October 2022 Published: 31 October 2022

Blood cell image segmentation is an important part of the field of computer-aided diagnosis. However, due to the low contrast, large differences in cell morphology and the scarcity of labeled images, the segmentation performance of cells cannot meet the requirements of an actual diagnosis. To address the above limitations, we present a deep learning-based approach to study cell segmentation on pathological images. Specifically, the algorithm selects UNet++ as the backbone network to extract multi-scale features. Then, the skip connection is redesigned to improve the degradation problem and reduce the computational complexity. In addition, the atrous spatial pyramid pooling (ASSP) is introduced to obtain cell image information features from each layer through different receptive domains. Finally, the multi-sided output fusion (MSOF) strategy is utilized to fuse the features of different semantic levels, so as to improve the accuracy of target segmentation. Experimental results on blood cell images for segmentation and classification (BCISC) dataset show that the proposed method has significant improvement in Matthew's correlation coefficient (Mcc), Dice and Jaccard values, which are better than the classical semantic segmentation network.
- cell,
- UNet++,
- atrous spatial pyramid pooling,
- image segmentation,
- U-Net
Citation: Kun Lan, Jianzhen Cheng, Jinyun Jiang, Xiaoliang Jiang, Qile Zhang. Modified UNet++ with atrous spatial pyramid pooling for blood cell image segmentation[J]. Mathematical Biosciences and Engineering, 2023, 20(1): 1420-1433. doi: 10.3934/mbe.2023064

Related Papers:

Abstract

Blood cell image segmentation is an important part of the field of computer-aided diagnosis. However, due to the low contrast, large differences in cell morphology and the scarcity of labeled images, the segmentation performance of cells cannot meet the requirements of an actual diagnosis. To address the above limitations, we present a deep learning-based approach to study cell segmentation on pathological images. Specifically, the algorithm selects UNet++ as the backbone network to extract multi-scale features. Then, the skip connection is redesigned to improve the degradation problem and reduce the computational complexity. In addition, the atrous spatial pyramid pooling (ASSP) is introduced to obtain cell image information features from each layer through different receptive domains. Finally, the multi-sided output fusion (MSOF) strategy is utilized to fuse the features of different semantic levels, so as to improve the accuracy of target segmentation. Experimental results on blood cell images for segmentation and classification (BCISC) dataset show that the proposed method has significant improvement in Matthew's correlation coefficient (Mcc), Dice and Jaccard values, which are better than the classical semantic segmentation network.

References

[1]	B. Dourthe, N. Shaikh, S. A. Pai, S. Fels, S. H. M. Brown, D. R. Wilson, et al., Automated segmentation of spinal muscles from upright open MRI using a multiscale pyramid 2D convolutional neural network, Sping, 47 (2022), 1179–1186. https://doi.org/10.1097/BRS.0000000000004308 doi: 10.1097/BRS.0000000000004308
[2]	K. Mariam, O. M. Afzal, W. Hussain, M. U. Javed, A. Kiyani, N. Rajpoot, et al., On smart gaze based annotation of histopathology images for training of deep convolutional neural networks, IEEE J. Biomed. Health Inf., 26 (2022), 3025–3036. https://doi.org/10.1109/JBHI.2022.3148944 doi: 10.1109/JBHI.2022.3148944
[3]	X. Y. Wei, Y. Y. Wang, L. Ge, B. Peng, Q. He, R. Wang, et al., Unsupervised convolutional neural network for motion estimation in ultrasound elastography, IEEE Trans. Ultrason. Ferroelectr. Freq. Control, 69 (2022), 2236–2247. https://doi.org/10.1109/TUFFC.2022.3171676 doi: 10.1109/TUFFC.2022.3171676
[4]	W. Ba, H. Wu, W. W. Chen, S. H. Wang, Z. Y. Zhang, X. J. Wei, et al., Convolutional neural network assistance significantly improves dermatologists' diagnosis of cutaneous tumours using clinical images, Eur. J. Cancer, 169 (2022), 156–165. https://doi.org/10.1016/j.ejca.2022.04.015 doi: 10.1016/j.ejca.2022.04.015
[5]	A. Iqbal, M. Sharif, M. A. Khan, W. Nisar, M. Alhaisoni, FF-UNet: a u-shaped deep convolutional neural network for multimodal biomedical image segmentation, Cognit. Comput., 14 (2022), 1287–1302. https://doi.org/10.1007/s12559-022-10038-y doi: 10.1007/s12559-022-10038-y
[6]	O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-assisted Intervention, (2015), 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
[7]	Z. Li, H. Zhang, Z. Li, Z. Ren, Residual-attention UNet++: a nested residual-attention U-Net for medical image segmentation, Appl. Sci., 12 (2022), 7149. https://doi.org/10.3390/app12147149 doi: 10.3390/app12147149
[8]	L. F. Yu, Z. Qin, Y. Ding, Z. G. Qin, MIA-UNet: multi-scale iterative aggregation U-Network for retinal vessel segmentation, Comput. Model. Eng. Sci., 129 (2021), 805–828. https://doi.org/10.32604/cmes.2021.017332 doi: 10.32604/cmes.2021.017332
[9]	Y. J. He, J. S. Li, S. Shen, K. Liu, K. K. Wong, T. C. He, et al., Image-to-image translation of label-free molecular vibrational images for a histopathological review using the UNet plus/seg-cGAN model, BioMed. Opt. Express, 13 (2020), 1924–1938. https://doi.org/10.1364/BOE.445319 doi: 10.1364/BOE.445319
[10]	Y. Zhang, X. Liu, S. Wa, Y. Liu, J. Kang, C. Lv, GenU-Net++: an automatic intracranial brain tumors segmentation algorithm on 3D image series with high performance, Symmetry, 13 (2021), 2395. https://doi.org/10.3390/sym13122395 doi: 10.3390/sym13122395
[11]	C. Wang, Z. Y. Zhao, Y. Yu, Fine retinal vessel segmentation by combining Nest U-net and patch-learning, Soft Comput., 25 (2021), 5519–5532. https://doi.org/10.1007/s00500-020-05552-w doi: 10.1007/s00500-020-05552-w
[12]	M. Lei, J. Li, M. Li, L. Zou, H. Yu, An improved UNet++ model for congestive heart failure diagnosis using short-term RR intervals, Diagnostics, 11 (2021), 534. https://doi.org/10.3390/diagnostics11030534 doi: 10.3390/diagnostics11030534
[13]	Y. Lu, X. J. Qin, H. Y. Fan, T. T. Lai, Z. Y. Li, WBC-Net: a white blood cell segmentation network based on UNet++ and ResNet, Appl. Soft Comput., 101 (2021), 107006. https://doi.org/10.1016/j.asoc.2020.107006 doi: 10.1016/j.asoc.2020.107006
[14]	Z. W. Zhou, M. M. R. Siddiquee, N. Tajbakhsh. J. M. Liang, UNet++: a nested U-Net architecture for medical image segmentation, in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, (2018), 3–11. https://doi.org/10.1007/978-3-030-00889-5_1
[15]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. https://doi.org/10.1109/CVPR.2016.90
[16]	S. X. Chan, C. Huang, C. Bai, W. L. Ding, S. Y. Chen, Res2-UNeXt: a novel deep learning framework for few-shot cell image segmentation, Multimedia Tools Appl., 81 (2021), 13275–13288. https://doi.org/10.1007/s11042-021-10536-5 doi: 10.1007/s11042-021-10536-5
[17]	V. Jumutc, D. Bliznuks, A. Lihachev, Multi-Path U-Net architecture for cell and colony-forming unit image segmentation, Sensors, 22 (2022), 990. https://doi.org/10.3390/s22030990 doi: 10.3390/s22030990
[18]	P. Thi Le, T. Pham, Y. C. Hsu, J. C. Wang, Convolutional blur attention network for cell nuclei segmentation, Sensors, 22 (2022), 1586. https://doi.org/10.3390/s22041586 doi: 10.3390/s22041586
[19]	M. Makem, A. Tiedeu, G. Kom, Y. Nkandeu, A robust algorithm for white blood cell nuclei segmentation, Multimedia Tools Appl., 81 (2022), 17849–17874. https://doi.org/10.1007/s11042-022-12285-5 doi: 10.1007/s11042-022-12285-5
[20]	B. Yang, M. X. Wu, W. Teizer, Modified UNet++ with attention gate for graphene identification by optical microscopy, Carbon, 195 (2022), 246–252. https://doi.org/10.1016/j.carbon.2022.03.035 doi: 10.1016/j.carbon.2022.03.035
[21]	S. Bhagat, M. Kokare, V. Haswani, P. Hambarde, R. Kamble, Eff-UNet++: a novel architecture for plant leaf segmentation and counting, Ecol. Inf., 68 (2022), 101583. https://doi.org/10.1016/j.ecoinf.2022.101583 doi: 10.1016/j.ecoinf.2022.101583
[22]	F. Hoorali, H. Khosravi, B. Moradib, Automatic Bacillus anthracis bacteria detection and segmentation in microscopic images using UNet++, J. Microbiol. Methods, 177 (2020), 106056. https://doi.org/10.1016/j.mimet.2020.106056 doi: 10.1016/j.mimet.2020.106056
[23]	W. Zhao, Y. Zhao, L. Feng, J. Tang, Attention enhanced serial Unet++ network for removing unevenly distributed haze, Electronics, 10 (2021), 2868. https://doi.org/10.3390/electronics10222868 doi: 10.3390/electronics10222868
[24]	H. Zhao, H. Zhang, X. Zheng, A multiscale attention-guided UNet++ with edge constraint for building extraction from high spatial resolution imagery, Appl. Sci., 12 (2022), 5960. https://doi.org/10.3390/app12125960 doi: 10.3390/app12125960
[25]	S. Safarov, T. K. Whangbo, A-DenseUNet: adaptive densely connected UNet for polyp segmentation in colonoscopy images with atrous convolution, Sensors, 21 (2021), 1441. https://doi.org/10.3390/s21041441 doi: 10.3390/s21041441
[26]	J. J. Li, Y. Han, M. Zhang, G. Li, B. H. Zhang, Multi-scale residual network model combined with global average pooling for action recognition, Multimedia Tools Appl., 81 (2022), 1375–1393. https://doi.org/10.1007/s11042-021-11435-5 doi: 10.1007/s11042-021-11435-5
[27]	R. L. Kumar, J. Kakarla, B. V. Isunuri, M. Singh, Multi-class brain tumor classification using residual network and global average pooling, Multimedia Tools Appl., 80 (2021), 13429–13438. https://doi.org/10.1007/s11042-020-10335-4 doi: 10.1007/s11042-020-10335-4
[28]	R. Arora, I. Saini, N. Sood, Multi-label segmentation and detection of COVID-19 abnormalities from chest radiographs using deep learning, Optik, 246 (2021), 167780. https://doi.org/10.1016/j.ijleo.2021.167780 doi: 10.1016/j.ijleo.2021.167780
[29]	X. M. Liu, S. C. Wang, Y. Zhang, D. Liu, W. Hu, Automatic fluid segmentation in retinal optical coherence tomography images using attention based deep learning, Neurocomputing, 452 (2021), 576–591. https://doi.org/10.1016/j.neucom.2020.07.143 doi: 10.1016/j.neucom.2020.07.143
[30]	T. D. T. Phan, S. H. Kim, H. J. Yang, G. S. Lee, Skin lesion segmentation by U-Net with adaptive skip connection and structural awareness, Appl. Sci., 11 (2021), 4528. https://doi.org/10.3390/app11104528 doi: 10.3390/app11104528
[31]	fpklipic, Dataset, 2019. Available from: https://github.com/fpklipic/BCISC.
[32]	M. Jiang, F. Zhai, J. Kong, A novel deep learning model DDU-net using edge features to enhance brain tumor segmentation on MR images, Artif. Intell. Med., 121 (2021), 102180. https://doi.org/10.1016/j.artmed.2021.102180 doi: 10.1016/j.artmed.2021.102180
[33]	A. Oulefki, S. Agaian, T. Trongtirakul, A. K. Laouar, Automatic COVID-19 lung infected region segmentation and measurement using CT-scans images, Pattern Recognit., 114 (2021), 107747. https://doi.org/10.1016/j.patcog.2020.107747 doi: 10.1016/j.patcog.2020.107747
[34]	Y. Y. Yang, C. Feng, R. F. Wang, Automatic segmentation model combining U-Net and level set method for medical images, Expert Syst. Appl., 153 (2020), 113419. https://doi.org/10.1016/j.eswa.2020.113419 doi: 10.1016/j.eswa.2020.113419
[35]	J. You, P. L. Yu, A. C. Tsang, E. L. Tsui, P. P. Woo, C. S. Lui, et al., 3D dissimilar-siamese-U-Net for hyperdense middle cerebral artery sign segmentation, Comput. Med. Imaging Graphics, 90 (2021), 101898. https://doi.org/10.1016/j.compmedimag.2021.101898 doi: 10.1016/j.compmedimag.2021.101898
[36]	S. Mohajerani, P. Saeedi, Cloud and cloud shadow segmentation for remote sensing imagery via filtered Jaccard loss function and parametric augmentation, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 14 (2021), 4254–4266. https://doi.org/10.1109/JSTARS.2021.3070786 doi: 10.1109/JSTARS.2021.3070786
[37]	V. S. Bochkov, L. Y. Kataeva, wUUNet: Advanced fully convolutional neural network for multiclass fire segmentation, Symmetry, 13 (2021), 98. https://doi.org/10.3390/sym13010098 doi: 10.3390/sym13010098
[38]	G. Huang, Z. Liu, V. Laurens, K. Q. Weinberger, Densely connected convolutional networks, in: IEEE Conference on Computer Vision and Pattern Recognition, (2017), 2261–2269. https://doi.org/10.1109/CVPR.2017.243
[39]	J. Long, E. Shelhamer, T Darrell, Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965
[40]	V. Badrinarayanan, A. Kendall, R. Cipolla, SegNet: a deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615 doi: 10.1109/TPAMI.2016.2644615
[41]	H. Huang, L. Lin, R. Tong, H. Hu, Q. Zhang, Y. Iwamoto, et al., UNet 3+: a full-scale connected UNet for medical image segmentation, in IEEE International Conference on Acoustics, Speech and Signal Processing, (2020), 1055–1059. https://doi.org/10.1109/ICASSP40776.2020.9053405

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)