Diagnosis of musculoskeletal abnormalities based on improved lightweight network for multiple model fusion

Zhigao Zeng; Changjie Song; Qiang Liu; Shengqiu Yi; Yanhui Zhu; Zhigao Zeng; Changjie Song; Qiang Liu; Shengqiu Yi; Yanhui Zhu

doi:10.3934/mbe.2024025

Mathematical Biosciences and Engineering

2024, Volume 21, Issue 1: 582-601. doi: 10.3934/mbe.2024025

Previous Article Next Article

Research article

Diagnosis of musculoskeletal abnormalities based on improved lightweight network for multiple model fusion

1.
School of Computer Science, Hunan University of Technology, Zhuzhou 412007, China
2.
Hunan Key Laboratory of Intelligent Information Perception and Processing Technology, Zhuzhou 412007, China

Received: 23 October 2023 Revised: 02 December 2023 Accepted: 10 December 2023 Published: 15 December 2023

This paper introduces a solution to address the intricacy of the model employed in the deep learning-based diagnosis of musculoskeletal abnormalities and the limitations observed in the performance of a single deep learning network model. The proposed approach involves the integration of an improved EfficientNet-B2 model with MobileNetV2, resulting in the creation of FusionNet. First, EfficientNet-B2 is combined with coordinate attention (CA) to obtain CA-EfficientNet-B2. Furthermore, aiming to minimize the model parameter count, we further enhanced the mobile inverted residual bottleneck convolution module (MBConv) employed for feature extraction in EfficientNet-B2, resulting in the development of CA-MBC-EfficientNet-B2. Next, the features extracted from CA-MBC-EfficientNet-B2 and MobileNetV2 are fused. Finally, the final diagnosis of musculoskeletal abnormalities was performed by using fully connected layers. The experimental results demonstrate that, first, compared to EfficientNet-B2, CA-MBC-EfficientNet-B2 not only significantly improves the diagnostic performance of musculoskeletal abnormalities, it also reduces the parameter count and storage space by 17%. Moreover, as compared to other models, FusionNet demonstrates remarkable performance in the area of anomaly diagnosis, particularly on the elbow dataset, achieving a precision of 92.93%, an AUC of 93.89% and an accuracy of 87.10%.
- diagnosis of musculoskeletal abnormalities,
- EfficientNet-B2,
- MobileNetV2,
- coordinate attention,
- mobile inverted residual bottleneck convolution
Citation: Zhigao Zeng, Changjie Song, Qiang Liu, Shengqiu Yi, Yanhui Zhu. Diagnosis of musculoskeletal abnormalities based on improved lightweight network for multiple model fusion[J]. Mathematical Biosciences and Engineering, 2024, 21(1): 582-601. doi: 10.3934/mbe.2024025

Related Papers:

Abstract

This paper introduces a solution to address the intricacy of the model employed in the deep learning-based diagnosis of musculoskeletal abnormalities and the limitations observed in the performance of a single deep learning network model. The proposed approach involves the integration of an improved EfficientNet-B2 model with MobileNetV2, resulting in the creation of FusionNet. First, EfficientNet-B2 is combined with coordinate attention (CA) to obtain CA-EfficientNet-B2. Furthermore, aiming to minimize the model parameter count, we further enhanced the mobile inverted residual bottleneck convolution module (MBConv) employed for feature extraction in EfficientNet-B2, resulting in the development of CA-MBC-EfficientNet-B2. Next, the features extracted from CA-MBC-EfficientNet-B2 and MobileNetV2 are fused. Finally, the final diagnosis of musculoskeletal abnormalities was performed by using fully connected layers. The experimental results demonstrate that, first, compared to EfficientNet-B2, CA-MBC-EfficientNet-B2 not only significantly improves the diagnostic performance of musculoskeletal abnormalities, it also reduces the parameter count and storage space by 17%. Moreover, as compared to other models, FusionNet demonstrates remarkable performance in the area of anomaly diagnosis, particularly on the elbow dataset, achieving a precision of 92.93%, an AUC of 93.89% and an accuracy of 87.10%.

References

[1]	J. Zhang, J. Z. Wang, Z. Yuan, E. S. Sobel, H. Jiang, Computer-aided classification of optical images for diagnosis of osteoarthritis in the finger joints, J. Xray. Sci. Technol., 19 (2011), 531–544. https://doi.org/10.3233/XST-2011-0312 doi: 10.3233/XST-2011-0312
[2]	M. Al-Ayyoub, D. Al-Zghool, Determining the type of long bone fractures in X-ray images, WSEAS Trans. Inf. Sci. Appl., 10 (2013), 261–270.
[3]	S. K. Mahendran, S. S. Baboo, An enhanced tibia fracture detection tool using image processing and classification fusion techniques in X-ray images, Glob. J. Comput. Sci. Technol., 11 (2011), 22–28.
[4]	H. Y. Chai, L. K. Wee, T. T. Swee, S. Hussain, Gray-level co-occurrence matrix bone fracture detection, WSEAS Trans. Syst., 10 (2011), 7–16.
[5]	M. Cohen, J. Puntonet, J. Sanchez, E. Kierszbaum, M. Crema, P. Soyer, et al., Artificial intelligence vs. radiologist: Accuracy of wrist fracture detection on radiographs, Eur. Radiol., 33 (2023), 3974–3983. https://doi.org/10.1007/s00330-022-09349-3 doi: 10.1007/s00330-022-09349-3
[6]	Y. Nam, Y. Choi, J. Kang, M. Seo, S. J. Heo, M. K. Lee, Diagnosis of nasal bone fractures on plain radiographs via convolutional neural networks, Sci. Rep., 12 (2022), 21510. https://doi.org/10.1038/s41598-022-26161-7 doi: 10.1038/s41598-022-26161-7
[7]	K. Oka, R. Shiode, Y. Yoshii, H. Tanaka, T. Iwahashi, T. Murase, Artificial intelligence to diagnosis distal radius fracture using biplane plain X-rays, J. Orthop. Res., 16 (2021), 694. https://doi.org/10.1186/s13018-021-02845-0 doi: 10.1186/s13018-021-02845-0
[8]	K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, arXiv: 1409.1556. https://arXiv.org/abs/1409.1556
[9]	M. He, X. Wang, Y. Zhao, A calibrated deep learning ensemble for abnormality detection in musculoskeletal radiographs, Sci. Rep., 11 (2021), 9097. https://doi.org/10.1038/s41598-021-88578-w doi: 10.1038/s41598-021-88578-w
[10]	G. Singh, D. Anand, W. Cho, G. P. Joshi, K. C. Son, Hybrid deep learning approach for automatic detection in musculoskeletal aadiographs, Biology, 11 (2022), 665. https://doi.org/10.3390/biology11050665 doi: 10.3390/biology11050665
[11]	P. H. Yi, T. K. Kim, J. Wei, J. Shin, F. K. Hui, H. I. Sair, et al., Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning, Pediatr. Radiol., 49 (2019), 1066–1070. https://doi.org/10.1007/s00247-019-04408-2 doi: 10.1007/s00247-019-04408-2
[12]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770–778.
[13]	J. Yao, Z. Guo, W. Yu, Enhanced deep residual network for bone classification and abnormality detection, Med. Phys., 49 (2022), 6914–6929. https://doi.org/10.1002/mp.15966 doi: 10.1002/mp.15966
[14]	C. T. Cheng, T. Y. Ho, T. Y. Lee, C. C. Chang, C. C. Chou, C. C. Chen, et al., Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs, Eur. Radiol., 29 (2019), 5469–5477. https://doi.org/10.1007/s00330-019-06167-y doi: 10.1007/s00330-019-06167-y
[15]	G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 2261–2269.
[16]	J. W. Choi, Y. J. Cho, S. Lee, J. Lee, S. Lee, Y. H. Choi, et al., Using a dual-input convolutional neural network for automated detection of pediatric supracondylar fracture on conventional radiography, Invest. Radiol., 55 (2020), 101–110. https://doi.org/10.1097/RLI.0000000000000615 doi: 10.1097/RLI.0000000000000615
[17]	C. T. Cheng, Y. Wang, H. W. Chen, P. M. Hsiao, C. N. Yeh, C. H. Hsieh, et al., A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs, Nat. Commun., 12 (2021), 1066. https://doi.org/10.1038/s41467-021-21311-3 doi: 10.1038/s41467-021-21311-3
[18]	Y. L. Thian, Y. Li, P. Jagmoha, D. Sia, V. E. Y. Chan, R. T. Tan, Convolutionalneural networks for automated fracture detection and localization on wrist radiographs, Radiol. Artif. Intell., 1 (2019), e180001. https://doi.org/10.1148/ryai.2019180001 doi: 10.1148/ryai.2019180001
[19]	C. Szegedy, S. Ioffe, V. Vanhoucke, A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, in Proceedings of the AAAI Conference on Artificial Intelligence, (2017), 4278–4284. https://doi.org/10.1609/aaai.v31i1.11231
[20]	S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, Adv. Neural Inf. Process. Syst., 39 (2015), 1137–1149.
[21]	P. Ye, S. Li, Z. Wang, S. Tian, Y. Luo, Z. Wu, et al., Development and validation of a deep learning-based model to distinguish acetabular fractures on pelvic anteroposterior radiographs, Front. Physiol., 14 (2023), 1146910. https://doi.org/10.3389/fphys.2023.1146910 doi: 10.3389/fphys.2023.1146910
[22]	X. Wang, Y. Wang, Composite attention residual U-Net for Rib fracture detection, Entropy, 25 (2023), 466. https://doi.org/10.3390/e25030466 doi: 10.3390/e25030466
[23]	O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, (2015), 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
[24]	L. Jin, J. Yang, K. Kuang, B. Ni, Y. Gao, Y. Sun, et al., Deep-learning-assisted detection and segmentation of rib fractures from CT scans: Development and validation of FracNet, EBioMedicine, 62 (2020), 103106. https://doi.org/10.1016/j.ebiom.2020.103106 doi: 10.1016/j.ebiom.2020.103106
[25]	A. Ghosh, D. Patton, S. Bose, M. K. Henry, M. Ouyang, H. Huang, et al., A patch-based deep learning approach for detecting rib fractures on frontal radiographs in young children, J. Digit Imaging, 36 (2023), 1302–1313. https://doi.org/10.1007/s10278-023-00793-1 doi: 10.1007/s10278-023-00793-1
[26]	T. Guo, C. Xu, S. He, B. Shi, C. Xu, D. Tao, Robust Student Network Learning, IEEE Trans. Neural Networks Learn. Syst., 31 (2020), 2455–2468. https://doi.org/10.1109/TNNLS.2019.2929114 doi: 10.1109/TNNLS.2019.2929114
[27]	L. Yang, G. Yuan, H. Wu, W. Qian, An ultra-lightweight detector with high accuracy and speed for aerial images, Math. Biosci. Eng., 20 (2023), 13947–13973. doi: 10.3934/mbe.2023621 doi: doi: 10.3934/mbe.2023621
[28]	E. Parcham, M. Fateh, HybridBranchNet: A novel structure for branch hybrid convolutional neural networks architecture, Neural Networks, 165 (2023), 77–93. https://doi.org/10.1016/j.neunet.2023.05.025 doi: 10.1016/j.neunet.2023.05.025
[29]	O. Eminaga, M. Abbas, J. Shen, M. Laurie, J. D. Brooks, J. C. Liao, et al., PlexusNet: A neural network architectural concept for medical image classification, Comput. Biol. Med., 154 (2023), 106594. https://doi.org/10.1016/j.compbiomed.2023.106594 doi: 10.1016/j.compbiomed.2023.106594
[30]	Z. Chen, J. Yang, L. Chen, H. Jiao, Garbage classification system based on improved ShuffleNet v2, Resour. Conserv. Recycl., 178 (2022), 106090. https://doi.org/10.1016/j.resconrec.2021.106090 doi: 10.1016/j.resconrec.2021.106090
[31]	M. Versaci, G. Angiulli, P. Crucitti, D. De Carlo, F. Laganà, D. Pellicanò, et al., A fuzzy similarity-based approach to classify numerically simulated and experimentally detected carbon fiber-reinforced polymer plate defects, Sensors, 22 (2022), 4232. https://doi.org/10.3390/s22114232 doi: 10.3390/s22114232
[32]	Z. Chen, J. Yang, C. Yang, BrightsightNet: A lightweight progressive low-light image enhancement network and its application in "Rainbow" maglev train, J. King Saud Univ-Com, 35 (2023), 101814. https://doi.org/10.1016/j.jksuci.2023.101814 doi: 10.1016/j.jksuci.2023.101814
[33]	P. Angelov, X. Gu, A cascade of deep learning fuzzy rule-based image classifier and SVM, in 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), (2017). https://doi.org/10.1109/SMC.2017.8122697
[34]	Z. Chen, J. Yang, L. Chen, Z. Feng, L. Jia, Efficient railway track region segmentation algorithm based on lightweight neural network and cross-fusion decoder, Autom. Constr., 155 (2023), 105069. https://doi.org/10.1016/j.autcon.2023.105069 doi: 10.1016/j.autcon.2023.105069
[35]	Z. Feng, J. Yang, Z. Chen, Z. Kang, LRseg: An efficient railway region extraction method based on lightweight encoder and self-correcting decoder, Expert Syst. Appl., 238 (2024), 122386. https://doi.org/10.1016/j.eswa.2023.122386 doi: 10.1016/j.eswa.2023.122386
[36]	M. Tan, Q. Le, EfficientNet: Rethinking model scaling for convolutional neural networks, in Proceedings of the 36th International Conference on Machine Learning(ICML), (2019), 6105–6114. https://arXiv.org/abs/1905.11946
[37]	M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L. C. Chen, MobileNetV2: Inverted residuals and linear bottlenecks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2018), 4510–4520.
[38]	Z. Chen, H. Guo, J. Yang, H. Jiao, Z. Feng, L. Chen, et al., Fast vehicle detection algorithm in traffic scene based on improved SSD, Measurement, 201 (2022), 111655. https://doi.org/10.1016/j.measurement.2022.111655 doi: 10.1016/j.measurement.2022.111655
[39]	P. Rajpurkar, J. Irvin, A. Bagul, D. Ding, T. Duan, H. Mehta, et al., Mura: Large dataset for abnormality detection in musculoskeletal radiographs, preprint, arXiv: 1712.06957.
[40]	K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, et al., Show, Attend and Tell: Neural omage caption generation with visual attention, in International Conference on Machine Learning(ICML), (2015), 2048–2057.
[41]	J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2018), 7132–7141.
[42]	Q. Hou, D. Zhou, J. Feng, Coordinate attention for efficient mobile network design, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), (2021), 13708–13717.
[43]	Q. Wang, B. Wu, P. Zhu, P. Li, Q. Hu, ECA-Net: Efficient channel attention for deep convolutional neural networks, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020).
[44]	A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, et al., MobileNets: Efficient convolutional neural networks for mobile vision applications, preprint, arXiv: 1704.04861.

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)