Computer Assisted Diagnosis (CAD) based on brain Magnetic Resonance Imaging (MRI) is a popular research field for the computer science and medical engineering. Traditional machine learning and deep learning methods were employed in the classification of brain MRI images in the previous studies. However, the current algorithms rarely take into consideration the influence of multi-scale brain connectivity disorders on some mental diseases. To improve this defect, a deep learning structure was proposed based on MRI images, which was designed to consider the brain's connections at different sizes and the attention of connections. In this work, a Multiscale View (MV) module was proposed, which was designed to detect multi-scale brain network disorders. On the basis of the MV module, the path attention module was also proposed to simulate the attention selection of the parallel paths in the MV module. Based on the two modules, we proposed a 3D Multiscale View Convolutional Neural Network with Attention (3D MVA-CNN) for classification of MRI images for mental disease. The proposed method outperformed the previous 3D CNN structures in the structural MRI data of ADHD-200 and the functional MRI data of schizophrenia. Finally, we also proposed a preliminary framework for clinical application using 3D CNN, and discussed its limitations on data accessing and reliability. This work promoted the assisted diagnosis of mental diseases based on deep learning and provided a novel 3D CNN method based on MRI data.
Citation: Zijian Wang, Yaqin Zhu, Haibo Shi, Yanting Zhang, Cairong Yan. A 3D multiscale view convolutional neural network with attention for mental disease diagnosis on MRI images[J]. Mathematical Biosciences and Engineering, 2021, 18(5): 6978-6994. doi: 10.3934/mbe.2021347
Computer Assisted Diagnosis (CAD) based on brain Magnetic Resonance Imaging (MRI) is a popular research field for the computer science and medical engineering. Traditional machine learning and deep learning methods were employed in the classification of brain MRI images in the previous studies. However, the current algorithms rarely take into consideration the influence of multi-scale brain connectivity disorders on some mental diseases. To improve this defect, a deep learning structure was proposed based on MRI images, which was designed to consider the brain's connections at different sizes and the attention of connections. In this work, a Multiscale View (MV) module was proposed, which was designed to detect multi-scale brain network disorders. On the basis of the MV module, the path attention module was also proposed to simulate the attention selection of the parallel paths in the MV module. Based on the two modules, we proposed a 3D Multiscale View Convolutional Neural Network with Attention (3D MVA-CNN) for classification of MRI images for mental disease. The proposed method outperformed the previous 3D CNN structures in the structural MRI data of ADHD-200 and the functional MRI data of schizophrenia. Finally, we also proposed a preliminary framework for clinical application using 3D CNN, and discussed its limitations on data accessing and reliability. This work promoted the assisted diagnosis of mental diseases based on deep learning and provided a novel 3D CNN method based on MRI data.
| [1] | S. G. Shamay-Tsoory, J. Aharon-Peretz, Dissociable prefrontal networks for cognitive and affective theory of mind: a lesion study, Neuropsychologia, 45 (2007), 3054-3067. |
| [2] | M. Hu, K. Sim, J. H. Zhou, X. Jiang, C. Guan, Brain MRI-based 3D Convolutional Neural Networks for Classification of Schizophrenia and Controls, Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS, (2020), 1742-1745 |
| [3] | E. Li, The application of BOLD-fMRI in cognitive neuroscience, J. Frontiers Comput. Sci. Technol., 2 (2008), 589-600. |
| [4] | K. J. Friston, L. Harrison, W. Penny, Dynamic causal modelling, Neuroimage, 19 (2003), 1273-1302. |
| [5] | F. Pereira, T. Mitchell, M. Botvinick, Machine learning classifiers and fMRI: a tutorial overview, Neuroimage, 45 (2009), S199-S209. |
| [6] |
S. Lemm, B. Blankertz, T. Dickhaus, K. R. Müller, Introduction to machine learning for brain imaging, Neuroimage, 56 (2011), 387-399. doi: 10.1016/j.neuroimage.2010.11.004
|
| [7] |
J. A. Suykens, J. Vandewalle, Least squares support vector machine classifiers, Neural Process. Lett., 9 (1999), 293-300. doi: 10.1023/A:1018628609742
|
| [8] | R. Hecht-Nielsen, Neural Networks for Perception, Academic Press, 1992. |
| [9] | A. Khazaee, A. Ebrahimzadeh, A. Babajani-Feremi, Application of advanced machine learning methods on resting-state fMRI network for identification of mild cognitive impairment and Alzheimer's disease, Brain Imaging Behav., 10 (2016), 799-817. |
| [10] | A. Al-Zubaidi, A. Mertins, M. Heldmann, K. Jauch-Chara, T. F. Münte, Machine learning based classification of resting-state fMRI features exemplified by metabolic state (hunger/satiety), Front. Hum. Neurosci., 13 (2019), 164. |
| [11] |
S. Patil, S. Choudhary, Deep convolutional neural network for chronic kidney disease prediction using ultrasound imaging, Bio-Algorithms Med. Syst., 17 (2021), 137-163 doi: 10.1515/bams-2020-0068
|
| [12] |
A. Dutta, T. Batabyal, M. Basu, S. T. Acton, An efficient convolutional neural network for coronary heart disease prediction, Expert Syst. Appl., 159 (2020), 113408. doi: 10.1016/j.eswa.2020.113408
|
| [13] | Y. Cao, Z. Wang, Z. Liu, Y. Li, X. Xiao, L. Sun, et al., Multiparameter synchronous measurement with IVUS images for intelligently diagnosing coronary cardiac disease, IEEE Trans. Instrum. Meas., (2020), 1-1 |
| [14] |
N. Zhang, G. Yang, Z. Gao, C. Xu, Y. Zhang, R. Shi, et al., Deep learning for diagnosis of chronic myocardial infarction on nonenhanced cardiac cine MRI, Radiology, 291 (2019), 606-617. doi: 10.1148/radiol.2019182304
|
| [15] |
Y. Jin, G. Yang, Y. Fang, R. Li, X. Xu, Y. Liu, et al., 3D PBV-Net: an automated prostate MRI data segmentation method, Comput. Biol. Med., 128 (2021), 104160. doi: 10.1016/j.compbiomed.2020.104160
|
| [16] | D. Driggs, I. Selby, M. Roberts, E. Gkrania-Klotsas, J. H. Rudd, G. Yang, et al., Machine learning for COVID-19 diagnosis and prognostication: lessons for amplifying the signal while reducing the noise, Radiol. Artif. Intell., 3 (2021), e210011. |
| [17] | S. Sarraf, G. Tofighi, Classification of alzheimer's disease using fmri data and deep learning convolutional neural networks, preprint, arXiv: 1603.08631 |
| [18] | S. Sarraf, D. D. DeSouza, J. Anderson, G. Tofighi, DeepAD: Alzheimer's disease classification via deep convolutional neural networks using MRI and fMRI, preprint, BioRxiv: 070441. |
| [19] |
Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, 86 (1998), 2278-2324. doi: 10.1109/5.726791
|
| [20] | Y. Zhao, Q. Dong, S. Zhang, W. Zhang, H. Chen, X. Jiang, et al., Automatic recognition of fMRI-derived functional networks using 3-D convolutional neural networks, IEEE Trans. Med. Imaging, 65 (2017), 1975-1984. |
| [21] |
L. Zou, J. Zheng, C. Miao, M. J. Mckeown, Z. J. Wang, 3D CNN based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural MRI, IEEE Access, 5 (2017), 23626-23636. doi: 10.1109/ACCESS.2017.2762703
|
| [22] |
Z. Wang, Y. Sun, Q. Shen, L. Cao, Dilated 3D convolutional neural networks for brain MRI data classification, IEEE Access, 7 (2019), 134388-134398. doi: 10.1109/ACCESS.2019.2941912
|
| [23] |
A. G. Garrity, G. D. Pearlson, K. McKiernan, D. Lloyd, K. A. Kiehl, V. D. Calhoun, Aberrant "default mode" functional connectivity in schizophrenia, Am. J. Psychiatry, 164 (2007), 450-457. doi: 10.1176/ajp.2007.164.3.450
|
| [24] | M.-E. Lynall, D. S. Bassett, R. Kerwin, P. J. McKenna, M. Kitzbichler, U. Muller, et al., Functional connectivity and brain networks in schizophrenia, J. Neurosci. Res., 30 (2010), 9477-9487. |
| [25] |
M. Murias, J. M. Swanson, R. Srinivasan, Functional connectivity of frontal cortex in healthy and ADHD children reflected in EEG coherence, Cereb. Cortex, 17 (2007), 1788-1799. doi: 10.1093/cercor/bhl089
|
| [26] | D. Fair, J. T. Nigg, S. Iyer, D. Bathula, K. L. Mills, N. U. Dosenbach, et al., Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data, Front. Syst. Neurosci., 6 (2013), 80. |
| [27] | S. Xie, R. Girshick, P. Dollár, Z. Tu, K. He, Aggregated residual transformations for deep neural networks, IEEE Comput. Conf. Comput. Vis. Pattern Recogn., (2017), 1492-1500 |
| [28] | J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, IEEE Comput. Conf. Comput. Vis. Pattern Recogn., (2018), 7132-7141 |
| [29] | K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, IEEE Comput. Conf. Comput. Vis. Pattern Recogn., (2016), 770-778 |
| [30] |
G. Yang, J. Chen, Z. Gao, S. Li, H. Ni, E. Angelini, et al., Simultaneous left atrium anatomy and scar segmentations via deep learning in multiview information with attention, Future Gener. Comput. Syst., 107 (2020), 215-228. doi: 10.1016/j.future.2020.02.005
|
| [31] |
Y. Liu, G. Yang, S. A. Mirak, M. Hosseiny, A. Azadikhah, X. Zhong, et al., Automatic prostate zonal segmentation using fully convolutional network with feature pyramid attention, IEEE Access, 7(2019), 163626-163632. doi: 10.1109/ACCESS.2019.2952534
|
| [32] |
W. Zhang, G. Yang, N. Zhang, L. Xu, X. Wang, Y. Zhang, et al., Multi-task learning with multi-view weighted fusion attention for artery-specific calcification analysis, Inf. Fusion, 71 (2021), 64-76. doi: 10.1016/j.inffus.2021.01.009
|
| [33] |
D. Zhang, G. Yang, S. Zhao, Y. Zhang, D. Ghista, H. Zhang, et al., Direct quantification of coronary artery stenosis through hierarchical attentive multi-view learning, IEEE Trans. Med. Imaging, 39 (2020), 4322-4334. doi: 10.1109/TMI.2020.3017275
|
| [34] | M. Yang, X. Xiao, Z. Liu, L. Sun, W. Guo, L. Cui, et al., Deep retinaNet for dynamic left ventricle detection in multiview echocardiography classification, Sci. Program, 2020 (2020), 7025403 |
| [35] |
M. Li, C. Wang, H. Zhang, G. Yang, MV-RAN: Multiview recurrent aggregation network for echocardiographic sequences segmentation and full cardiac cycle analysis, Comput. Biol. Med., 120 (2020), 103728. doi: 10.1016/j.compbiomed.2020.103728
|
| [36] | M. R. Brown, G. S. Sidhu, R. Greiner, N. Asgarian, M. Bastani, P. H. Silverstone, et al., ADHD-200 global competition: diagnosing ADHD using personal characteristic data can outperform resting state fMRI measurements, Front. Syst. Neurosci., 6 (2012), 69. |
| [37] | W. Liu, K. Zeng, SparseNet: A sparse denseNet for image classification, preprint, arXiv: 1804.05340 |
| [38] | C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, IEEE Comput. Conf. Comput. Vis. Pattern Recogn., (2016), 2818-2826 |
| [39] | B. Sen, N. C. Borle, R. Greiner, M. R. Brown, A general prediction model for the detection of ADHD and Autism using structural and functional MRI, PloS One, 13 (2018), e0194856. |
| [40] | S. Ghiassian, R. Greiner, P. Jin, M. R. Brown, Using functional or structural magnetic resonance images and personal characteristic data to identify ADHD and autism, PloS One, 11 (2016), e0166934. |
| [41] |
F. Raschke, T. R. Barrick, T. L. Jones, G. Yang, X. Ye, F. A. Howe, Tissue-type mapping of gliomas, NeuroImage: Clin., 21 (2019), 101648. doi: 10.1016/j.nicl.2018.101648
|
Figures(8) / Tables(3)
Zijian Wang, Yaqin Zhu, Haibo Shi, Yanting Zhang, Cairong Yan. A 3D multiscale view convolutional neural network with attention for mental disease diagnosis on MRI images[J]. Mathematical Biosciences and Engineering, 2021, 18(5): 6978-6994. doi: 10.3934/mbe.2021347
DownLoad: