mfeeU-Net: A multi-scale feature extraction and enhancement U-Net for automatic liver segmentation from CT Images

Jun Liu; Zhenhua Yan; Chaochao Zhou; Liren Shao; Yuanyuan Han; Yusheng Song; Jun Liu; Zhenhua Yan; Chaochao Zhou; Liren Shao; Yuanyuan Han; Yusheng Song

doi:10.3934/mbe.2023336

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 5: 7784-7801. doi: 10.3934/mbe.2023336

Previous Article Next Article

Research article Special Issues

mfeeU-Net: A multi-scale feature extraction and enhancement U-Net for automatic liver segmentation from CT Images

1.
Department of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
2.
Department of Radiology, Feinberg School of Medicine, Northwestern University, Chicago 60611, Illinois, U.S
3.
Interventional Radiology, The People's Hospital of Ganzhou, Ganzhou 34100, Jiangxi, China

Academic Editor: Sokratis Makrogiannis

Received: 05 December 2022 Revised: 12 February 2023 Accepted: 15 February 2023 Published: 21 February 2023

Medical image segmentation of the liver is an important prerequisite for clinical diagnosis and evaluation of liver cancer. For automatic liver segmentation from Computed Tomography (CT) images, we proposed a Multi-scale Feature Extraction and Enhancement U-Net (mfeeU-Net), incorporating Res2Net blocks, Squeeze-and-Excitation (SE) blocks, and Edge Attention (EA) blocks. The Res2Net blocks which are conducive to extracting multi-scale features of the liver were used as the backbone of the encoder, while the SE blocks were also added to the encoder to enhance channel information. The EA blocks were introduced to skip connections between the encoder and the decoder, to facilitate the detection of blurred liver edges where the intensities of nearby organs are close to the liver. The proposed mfeeU-Net was trained and evaluated using a publicly available CT dataset of LiTS2017. The average dice similarity coefficient, intersection-over-union ratio, and sensitivity of the mfeeU-Net for liver segmentation were 95.32%, 91.67%, and 95.53%, respectively, and all these metrics were better than those of U-Net, Res-U-Net, and Attention U-Net. The experimental results demonstrate that the mfeeU-Net can compete with and even outperform recently proposed convolutional neural networks and effectively overcome challenges, such as discontinuous liver regions and fuzzy liver boundaries.
- liver segmentation,
- U-Net,
- multi-scale,
- feature extraction and enhancement,
- Res2Net,
- squeeze-and-excitation,
- edge attention
Citation: Jun Liu, Zhenhua Yan, Chaochao Zhou, Liren Shao, Yuanyuan Han, Yusheng Song. mfeeU-Net: A multi-scale feature extraction and enhancement U-Net for automatic liver segmentation from CT Images[J]. Mathematical Biosciences and Engineering, 2023, 20(5): 7784-7801. doi: 10.3934/mbe.2023336

Related Papers:

Abstract

Medical image segmentation of the liver is an important prerequisite for clinical diagnosis and evaluation of liver cancer. For automatic liver segmentation from Computed Tomography (CT) images, we proposed a Multi-scale Feature Extraction and Enhancement U-Net (mfeeU-Net), incorporating Res2Net blocks, Squeeze-and-Excitation (SE) blocks, and Edge Attention (EA) blocks. The Res2Net blocks which are conducive to extracting multi-scale features of the liver were used as the backbone of the encoder, while the SE blocks were also added to the encoder to enhance channel information. The EA blocks were introduced to skip connections between the encoder and the decoder, to facilitate the detection of blurred liver edges where the intensities of nearby organs are close to the liver. The proposed mfeeU-Net was trained and evaluated using a publicly available CT dataset of LiTS2017. The average dice similarity coefficient, intersection-over-union ratio, and sensitivity of the mfeeU-Net for liver segmentation were 95.32%, 91.67%, and 95.53%, respectively, and all these metrics were better than those of U-Net, Res-U-Net, and Attention U-Net. The experimental results demonstrate that the mfeeU-Net can compete with and even outperform recently proposed convolutional neural networks and effectively overcome challenges, such as discontinuous liver regions and fuzzy liver boundaries.

References

[1]	R. L. Siegel, K. D. Miller, H. E. Fuchs, A. Jemal, Cancer statistics, 2021, CA Cancer J. Clin., 71 (2021), 7–33. https://doi.org/10.3322/caac.21654
[2]	S. Gul, M. S. Khan, A. Bibi, A. Khandakar, M. A. Ayari, M. E. H. Chowdhury, Deep learning techniques for liver and liver tumor segmentation: A review, Comput. Biol. Med., 147 (2022), 105620. https://doi.org/10.1016/j.compbiomed.2022.105620 doi: 10.1016/j.compbiomed.2022.105620
[3]	X. Shu, Y. Yang, B. Wu, Adaptive segmentation model for liver CT images based on neural network and level set method, Neurocomputing, 453 (2021), 438–452. https://doi.org/10.1016/j.neucom.2021.01.081 doi: 10.1016/j.neucom.2021.01.081
[4]	L. Soler, H. Delingette, G. Malandain, J. Montagnat, N. Ayache, C. Koehl, et al., Fully automatic anatomical, pathological, and functional segmentation from CT scans for hepatic surgery, Comput. Aided Surg., 6 (2010) 131–142. https://doi.org/10.3109/10929080109145999 doi: 10.3109/10929080109145999
[5]	X. Lu, J. Wu, X. Ren, B. Zhang, Y. Li, The study and application of the improved region growing algorithm for liver segmentation, Optik, 125 (2014), 2142–2147. https://doi.org/10.1016/j.ijleo.2013.10.049 doi: 10.1016/j.ijleo.2013.10.049
[6]	J. Wang, Y. Cheng, C. Guo, Y. Wang, S. Tamura, Shape-intensity prior level set combining probabilistic atlas and probability map constrains for automatic liver segmentation from abdominal CT images, Int. J. Comput. Assist. Radiol. Surg., 11 (2016), 817–826. https://doi.org/10.1007/s11548-015-1332-9 doi: 10.1007/s11548-015-1332-9
[7]	J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 3431–3440. https://doi.org/10.1109/TPAMI.2016.2572683
[8]	O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, (2015), 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
[9]	Z. Zhang, Q. Liu, Y. Wang, Road extraction by deep residual u-net, IEEE Geosci. Remote Sens. Lett., 15 (2018), 749–753. https://doi.org/10.1109/LGRS.2018.2802944 doi: 10.1109/LGRS.2018.2802944
[10]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. https://doi.org/10.1109/CVPR.2016.90
[11]	O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, et al., Attention U-net: Learning where to look for the pancreas, preprint, arXiv: 1804.03999. https://doi.org/10.48550/arXiv.1804.03999
[12]	C. Li, Y. Tan, W. Chen, X. Luo, Y. Gao, X. Jia, et al., Attention Unet++: A nested attention-aware U-net for liver CT image segmentation, in 2020 IEEE International Conference on Image Processing (ICIP), (2020), 345–349. https://doi.org/10.1109/ICIP40778.2020.9190761
[13]	Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, J. Liang, UNet++: A Nested U-net architecture for medical image segmentation, in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, (2018), 3–11.
[14]	D. T. Kushnure, S. N. Talbar, MS-UNet: A multi-scale UNet with feature recalibration approach for automatic liver and tumor segmentation in CT images, Comput. Med. Imaging Graph., 89 (2021), 101885. https://doi.org/10.1016/j.compmedimag.2021.101885 doi: 10.1016/j.compmedimag.2021.101885
[15]	J. Wang, X. Zhang, P. Lv, L. Zhou, H. Wang, EAR-U-Net: EfficientNet and attention-based residual U-Net for automatic liver segmentation in CT, preprint, arXiv: 2110.01014. https://doi.org/10.48550/arXiv.2110.01014
[16]	P. Lv, J. Wang, H. Wang, 2.5D lightweight RIU-Net for automatic liver and tumor segmentation from CT, Biomed. Signal Process. Control, 75 (2022), 103567. https://doi.org/10.1016/j.bspc.2022.103567 doi: 10.1016/j.bspc.2022.103567
[17]	J. Wang, X. Zhang, L. Guo, C. Shi, S. Tamura, Multi-scale attention and deep supervision-based 3D UNet for automatic liver segmentation from CT, Math. Biosci. Eng., 20 (2023), 1297–1316. https://doi.org/10.3934/mbe.2023059 doi: 10.3934/mbe.2023059
[18]	T. Fan, G. Wang, X. Wang, Y. Li, H. Wang, MSN-Net: A multi-scale context nested U-Net for liver segmentation, Signal Image Video P, 15 (2021), 1089–1097. https://doi.org/10.1007/s11760-020-01835-9 doi: 10.1007/s11760-020-01835-9
[19]	P. Lv, J. Wang, X. Zhang, C. Ji, L. Zhou, H. Wang, An improved residual U-Net with morphological-based loss function for automatic liver segmentation in computed tomography, Math. Biosci. Eng., 19 (2022), 1426–1447. https://doi.org/10.3934/mbe.2022066 doi: 10.3934/mbe.2022066
[20]	J. D. L. Araújo, L. B. da Cruz, J. O. B. Diniz, J. L. Ferreira, A. C. Silva, A. C. de Paiva, et al., Liver segmentation from computed tomography images using cascade deep learning, Comput. Biol. Med., 140 (2022), 105095. https://doi.org/10.1016/j.compbiomed.2021.105095 doi: 10.1016/j.compbiomed.2021.105095
[21]	H. Huang, L. Lin, R. Tong, H. Hu, Q. Zhang, Y. Iwamoto, et al., Unet 3+: A full-scale connected Unet for medical image segmentation, in ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2020), 1055–1059. https://doi.org/10.1109/ICASSP40776.2020.9053405
[22]	S. Sun, Z. Cao, D. Liao, R. Lv, A magnified adaptive feature pyramid network for automatic microaneurysms detection, Comput. Biol. Med., 139 (2021), 105000. https://doi.org/10.1016/j.compbiomed.2021.105000 doi: 10.1016/j.compbiomed.2021.105000
[23]	S. H. Gao, M. M. Cheng, K. Zhao, X. Y. Zhang, M. H. Yang, P. Torr, Res2Net: A new multi-scale backbone architecture, IEEE Trans. Pattern Anal. Mach. Intell., 43 (2019), 652–662. https://doi.org/10.1109/TPAMI.2019.2938758 doi: 10.1109/TPAMI.2019.2938758
[24]	J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2018), 7132–7141. https://doi.org/10.1109/CVPR.2018.00745
[25]	A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Commun. ACM, 60 (2017), 84–90. https://doi.org/10.1145/3065386 doi: 10.1145/3065386
[26]	C. Szegedy, S. Ioffe, V. Vanhoucke, A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, in Proceedings of the AAAI Conference on Artificial Intelligence, 31 (2017). https://doi.org/10.1609/aaai.v31i1.11231
[27]	J. Wang, P. Lv, H. Wang, C. Shi, SAR-U-Net: Squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in computed tomography, Comput. Methods Programs Biomed., 208 (2021), 106268. https://doi.org/10.1016/j.cmpb.2021.106268 doi: 10.1016/j.cmpb.2021.106268
[28]	D. T. Kushnure, S. Tyagi, S. N. Talbar, LiM-Net: Lightweight multi-level multiscale network with deep residual learning for automatic liver segmentation in CT images, Biomed. Signal Process. Control, 80 (2023), 104305. https://doi.org/10.1016/j.bspc.2022.104305 doi: 10.1016/j.bspc.2022.104305
[29]	T. Fan, G. Wang, Y. Li, H. Wang, MA-Net: A multi-scale attention network for liver and tumor segmentation, IEEE Access, 8 (2020), 179656–179665. https://doi.org/10.1109/ACCESS.2020.3025372 doi: 10.1109/ACCESS.2020.3025372
[30]	D. P. Fan, G. P. Ji, T. Zhou, G. Chen, H. Fu, J. Shen, et al., PraNet: Parallel reverse attention network for polyp segmentation, in Medical Image Computing and Computer Assisted Intervention—MICCAI 2020, (2020), 263–273. https://doi.org/10.1007/978-3-030-59725-2_26
[31]	C. Y. Lee, S. Xie, P. Gallagher, Z. Zhang, Z. Tu, Deeply-supervised nets, in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, 38 (2015), 562–570. https://doi.org/10.48550/arXiv.1409.5185
[32]	P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou, et al., The liver tumor segmentation benchmark (lits), Med. Image Anal., 84 (2023), 102680. https://doi.org/10.1016/j.media.2022.102680 doi: 10.1016/j.media.2022.102680
[33]	C. Zhang, Q. Hua, Y. Chu, P. Wang, Liver tumor segmentation using 2.5D UV-Net with multi-scale convolution, Comput. Biol. Med., 133 (2021), 104–424. https://doi.org/10.1016/j.compbiomed.2021.104424 doi: 10.1016/j.compbiomed.2021.104424
[34]	Q. Jin, Z. Meng, C. Sun, H. Cui, R. Su, RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans, Front. Bioeng. Biotechnol., 8 (2020), 1471. https://doi.org/10.3389/fbioe.2020.605132 doi: 10.3389/fbioe.2020.605132
[35]	J. Li, X. Ou, N. Shen, J. Sun, J. Ding, J. Zhang, et al., Study on strategy of CT image sequence segmentation for liver and tumor based on U-Net and Bi-ConvLSTM, Expert Syst. Appl., 180 (2021), 115008. https://doi.org/10.1016/j.eswa.2021.115008 doi: 10.1016/j.eswa.2021.115008
[36]	R. Bi, C. Ji, Z. Yang, M. Qiao, P. Lv, H. Wang, Residual based attention-Unet combing DAC and RMP modules for automatic liver tumor segmentation in CT, Math. Biosci. Eng., 19 (2022), 4703–4718. https://doi.org/10.3934/mbe.2022219 doi: 10.3934/mbe.2022219
[37]	S. Shao, X. Zhang, R. Cheng, C. Deng, Semantic segmentation method of 3D liver image based on contextual attention model, in 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), (2021), 3042–3049. https://doi.org/10.1109/SMC52423.2021.9659018

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)