Second-order ResU-Net for automatic MRI brain tumor segmentation

Ning Sheng; Dongwei Liu; Jianxia Zhang; Chao Che; Jianxin Zhang; Ning Sheng; Dongwei Liu; Jianxia Zhang; Chao Che; Jianxin Zhang

doi:10.3934/mbe.2021251

Mathematical Biosciences and Engineering

2021, Volume 18, Issue 5: 4943-4960. doi: 10.3934/mbe.2021251

Previous Article Next Article

Research article Special Issues

Second-order ResU-Net for automatic MRI brain tumor segmentation

1.
Key Lab of Advanced Design and Intelligent Computing (Ministry of Education), Dalian University, Dalian 116622, China
2.
School of Computer Science and Engineering, Dalian Minzu University, Dalian 116600, China
3.
School of Intelligent Engineering, Henan Institute of Technology, Xinxiang 453003, China

Received: 08 April 2021 Accepted: 19 May 2021 Published: 07 June 2021

Tumor segmentation using magnetic resonance imaging (MRI) plays a significant role in assisting brain tumor diagnosis and treatment. Recently, U-Net architecture with its variants have become prevalent in the field of brain tumor segmentation. However, the existing U-Net models mainly exploit coarse first-order features for tumor segmentation, and they seldom consider the more powerful second-order statistics of deep features. Therefore, in this work, we aim to explore the effectiveness of second-order statistical features for brain tumor segmentation application, and further propose a novel second-order residual brain tumor segmentation network, i.e., SoResU-Net. SoResU-Net utilizes a number of second-order modules to replace the original skip connection operations, thus augmenting the series of transformation operations and increasing the non-linearity of the segmentation network. Extensive experimental results on the BraTS 2018 and BraTS 2019 datasets demonstrate that SoResU-Net outperforms its baseline, especially on core tumor and enhancing tumor segmentation, illuminating the effectiveness of second-order statistical features for the brain tumor segmentation application.
- brain tumor segmentation,
- second-order statistics,
- U-Net,
- residual module
Citation: Ning Sheng, Dongwei Liu, Jianxia Zhang, Chao Che, Jianxin Zhang. Second-order ResU-Net for automatic MRI brain tumor segmentation[J]. Mathematical Biosciences and Engineering, 2021, 18(5): 4943-4960. doi: 10.3934/mbe.2021251

Related Papers:

Abstract

Tumor segmentation using magnetic resonance imaging (MRI) plays a significant role in assisting brain tumor diagnosis and treatment. Recently, U-Net architecture with its variants have become prevalent in the field of brain tumor segmentation. However, the existing U-Net models mainly exploit coarse first-order features for tumor segmentation, and they seldom consider the more powerful second-order statistics of deep features. Therefore, in this work, we aim to explore the effectiveness of second-order statistical features for brain tumor segmentation application, and further propose a novel second-order residual brain tumor segmentation network, i.e., SoResU-Net. SoResU-Net utilizes a number of second-order modules to replace the original skip connection operations, thus augmenting the series of transformation operations and increasing the non-linearity of the segmentation network. Extensive experimental results on the BraTS 2018 and BraTS 2019 datasets demonstrate that SoResU-Net outperforms its baseline, especially on core tumor and enhancing tumor segmentation, illuminating the effectiveness of second-order statistical features for the brain tumor segmentation application.

References

[1]	J. Liu, M. Li, J. Wang, F. Wu, Y. Pan, A survey of MRI-based brain tumor segmentation methods, Tsinghua Sci. Technol., 19 (2014), 578-595. doi: 10.1109/TST.2014.6961028
[2]	P. Y. Wen, D. R. Macdonald, D. A. Reardon, Updated response assessment criteria for high-grade gliomas: Response assessment in neuro-oncology working group, J. Clin. Oncol., 28 (2010), 1963-1972. doi: 10.1200/JCO.2009.26.3541
[3]	K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, arXiv: 1409.1556.
[4]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, IEEE Computer Society, (2016), 770-778.
[5]	G. Huang, Z. Liu, L. van der Maaten, K. Q. Weinberger, Densely connected convolutional networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, IEEE Computer Society, (2017), 2261-2269.
[6]	B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al., The multimodal brain tumor image segmentation benchmark (BRATS), IEEE Transact. Med. Imag., 34 (2014), 1993-2024.
[7]	S. Pereira, A. Pinto, V. Alves, C. A. Silva, Brain tumor segmentation using convolutional neural networks in MRI images, IEEE Transact. Med. Imag., 35 (2016), 1240-1251. doi: 10.1109/TMI.2016.2538465
[8]	P. Moeskops, M. A. Viergever, A. M. Mendrik, L. S. De Vries, M. J. Benders, I. Išgum, Automatic segmentation of MR brain images with a convolutional neural network, IEEE Transact. Med. Imag., 35 (2016), 1252-1261. doi: 10.1109/TMI.2016.2548501
[9]	J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, IEEE Computer Society, (2015), 3431-3440.
[10]	O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015-18th International Conference Munich, Lecture Notes in Computer Science, 9351, Springer, (2015), 234-241.
[11]	H. Dong, G. Yang, F. Liu, Y. Mo, Y. Guo, Automatic brain tumor detection and segmentation using u-net based fully convolutional networks, in Medical Image Understanding and Analysis, 21st Annual Conference, MIUA 2017, Communications in Computer and Information Science, 723, Springer, (2017), 506-517.
[12]	N. M. Aboelenein, P. Songhao, A. Koubaa, A. Afifi, HTTU-Net: Hybrid Two Track U-Net for automatic brain tumor segmentation, IEEE Access, 8 (2020), 101406-101415. doi: 10.1109/ACCESS.2020.2998601
[13]	X. Cheng, Z. Jiang, Q. Sun, Memory-Efficient Cascade 3D U-Net for Brain Tumor Segmentation, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part I, Lecture Notes in Computer Science, Springer, Cham, (2019), 242-253.
[14]	F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, K. H. Maier-Hein, No new-net, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11384, Springer, (2018), 234-244.
[15]	A. Myronenko, 3D MRI brain tumor segmentation using autoencoder regularization, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries--4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11384, Springer, (2018), 311-320.
[16]	Z. Jiang, C. Ding, M. Liu, Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part I, Lecture Notes in Computer Science, 11992, Springer, (2019), 231-241.
[17]	P. Li, J. Xie, Q. Wang, Is second-order information helpful for large-scale visual recognition?, in IEEE International Conference on Computer Vision, ICCV 2017, IEEE Computer Society, (2017), 2070-2078.
[18]	M. Cimpoi, S. Maji, A. Vedaldi, Deep filter banks for texture recognition and segmentation, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, IEEE Computer Society, (2015), 3828-3836.
[19]	T. Y. Lin, A. Roy Chowdhury, S. Maji, Bilinear CNN models for fine-grained visual recognition, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, IEEE Computer Society, (2015), 1449-1457.
[20]	B. Chen, W. Deng, J. Hu, Mixed high-order attention network for person re-identification, in IEEE/CVF International Conference on Computer Vision, ICCV 2019, IEEE, (2019), 371-381.
[21]	Q. Wang, P. Li, L. Zhang, G2DeNet: Global Gaussian distribution embedding network and its application to visual recognition, in 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR2017, IEEE Computer Society, (2017), 2730-2739.
[22]	A. Cherian, P. Koniusz, S. Gould, Higher-order pooling of CNN features via kernel linearization for action recognition in 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, (2017), 130-138.
[23]	N. Gordillo, E. Montseny, P. Sobrevilla, State of the art survey on MRI brain tumor segmentation, Magnet. Reson. Imag., 31 (2013), 1426-1438. doi: 10.1016/j.mri.2013.05.002
[24]	C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, M. J. Cardoso, Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations, in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Lecture Notes in Computer Science, 10553, Springer, (2017), 240-248.
[25]	A. Kermi, I. Mahmoudi, M. T. Khadir, Deep convolutional neural networks using U-Net for automatic brain tumor segmentation in multimodal MRI volumes, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, Brain-Les 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11384, Springer, (2018), 37-48.
[26]	M. Bhalerao, S. Thakur, Brain Tumor Segmentation Based on 3D Residual U-Net, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11993, Springer, (2019), 218-225.
[27]	Y. L. Tai, S. J. Huang, C. C. Chen, H. S. Lu, Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi-Dirac Correction Functions, Entropy, 23 (2021), 223. doi: 10.3390/e23020223
[28]	U. Baid, N. A. Shah, S. Talbar, Brain Tumor Segmentation with Cascaded Deep Convolutional Neural Network, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11993. Springer, (2019), 90-98.
[29]	M. U. Rehman, S. B. Cho, J. Kim, K. T. Chong, BrainSeg-Net: Brain Tumor MR Image Segmentation via Enhanced Encoder-Decoder Network, Diagnostics, 11 (2021), 169. doi: 10.3390/diagnostics11020169
[30]	J. Zhang, Z. Jiang, J. Dong, Attention Gate ResU-Net for automatic MRI brain tumor segmentation, IEEE Access, 8 (2020), 58533-58545. doi: 10.1109/ACCESS.2020.2983075
[31]	M. Amian, M. Soltaninejad, Multi-resolution 3D CNN for MRI Brain Tumor Segmentation and Survival Prediction, inBrainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11993, Springer, (2019), 221-230.
[32]	K. Hu, Q. Gan, Y. Zhang, S. Deng, F. Xiao, W. Huang et al., Brain tumor segmentation using multi-Cascaded Convolutional Neural Networks and Conditional Random Field, IEEE Access, 7 (2019), 92615-92629. doi: 10.1109/ACCESS.2019.2927433
[33]	M. Marcinkiewicz, J. Nalepa, P. R. Lorenzo, W. Dudzik, G. Mrukwa, Automatic Brain Tumor Segmentation Using a Two-Stage Multi-Modal FCNN, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, (2018), 13-24.
[34]	S. Chandra, M. Vakalopoulou, L. Fidon, E. Battistella, T. Estienne, R. Sunet, et al., Context aware 3D CNNs for brain tumor segmentation, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Revised Selected Papers, Part II, Lecture Notes in Computer Science, 11384, Springer, (2018), 299-310.
[35]	U. Baid, S. Talbar, S. Rane, S. Gupta, M. H. Thakur, A. Moiyadi, et al., A novel approach for fully automatic intra-tumor segmentation with 3D U-Net architecture for gliomas, Frontiers Comput. Neurosci., 14 (2020), 10.
[36]	Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, O. Ronneberger, 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation, in MICCAI: International Conference on Medical Image Computing and Computer-Assisted Intervention, 19th International Conference, Athens, 2016, Proceedings, Part II, Lecture Notes in Computer Science, 9901, Springer, (2016), 424-432.

Reader Comments

Your name:*

Email:*
© 2021 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)