EMG gesture signal analysis towards diagnosis of upper limb using dual-pathway convolutional neural network

Hafiz Ghulam Murtza Qamar; Muhammad Farrukh Qureshi; Zohaib Mushtaq; Zubariah Zubariah; Muhammad Zia ur Rehman; Nagwan Abdel Samee; Noha F. Mahmoud; Yeong Hyeon Gu; Mohammed A. Al-masni; Hafiz Ghulam Murtza Qamar; Muhammad Farrukh Qureshi; Zohaib Mushtaq; Zubariah Zubariah; Muhammad Zia ur Rehman; Nagwan Abdel Samee; Noha F. Mahmoud; Yeong Hyeon Gu; Mohammed A. Al-masni

doi:10.3934/mbe.2024252

Mathematical Biosciences and Engineering

2024, Volume 21, Issue 4: 5712-5734. doi: 10.3934/mbe.2024252

Previous Article Next Article

Research article Special Issues

EMG gesture signal analysis towards diagnosis of upper limb using dual-pathway convolutional neural network

1.
School of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei 066104, China
2.
Department of Electrical Engineering, Riphah International University, Islamabad 44000, Pakistan
3.
Department of Electrical, Electronics and Computer Systems, College of Engineering and Technology, University of Sargodha, Sargodha 40100, Pakistan
4.
Department of Physiotherapy, Isfandyar Bukhari Civil Hospital, District Headquarter Hospital, Attock 43600, Pakistan
5.
Department of Biomedical Engineering, Riphah International University, Islamabad 44000, Pakistan
6.
Department of Health Science and Technology, Aalborg University, Aalborg 9220, Denmark
7.
Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
8.
Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh 11671, Saudi Arabia
9.
Department of Artificial Intelligence Data Science, College of Software & Convergence Technology, Sejong University, Seoul 05006, Republic of Korea

Academic Editor: Giuseppe Ciaburro

Received: 09 November 2023 Revised: 02 February 2024 Accepted: 27 February 2024 Published: 24 April 2024

This research introduces a novel dual-pathway convolutional neural network (DP-CNN) architecture tailored for robust performance in Log-Mel spectrogram image analysis derived from raw multichannel electromyography signals. The primary objective is to assess the effectiveness of the proposed DP-CNN architecture across three datasets (NinaPro DB1, DB2, and DB3), encompassing both able-bodied and amputee subjects. Performance metrics, including accuracy, precision, recall, and F1-score, are employed for comprehensive evaluation. The DP-CNN demonstrates notable mean accuracies of 94.93 ± 1.71% and 94.00 ± 3.65% on NinaPro DB1 and DB2 for healthy subjects, respectively. Additionally, it achieves a robust mean classification accuracy of 85.36 ± 0.82% on amputee subjects in DB3, affirming its efficacy. Comparative analysis with previous methodologies on the same datasets reveals substantial improvements of 28.33%, 26.92%, and 39.09% over the baseline for DB1, DB2, and DB3, respectively. The DP-CNN's superior performance extends to comparisons with transfer learning models for image classification, reaffirming its efficacy. Across diverse datasets involving both able-bodied and amputee subjects, the DP-CNN exhibits enhanced capabilities, holding promise for advancing myoelectric control.
- electromyography,
- nina pro,
- convolutional neural network,
- log mel spectrogram,
- upper limb,
- gesture recognition
Citation: Hafiz Ghulam Murtza Qamar, Muhammad Farrukh Qureshi, Zohaib Mushtaq, Zubariah Zubariah, Muhammad Zia ur Rehman, Nagwan Abdel Samee, Noha F. Mahmoud, Yeong Hyeon Gu, Mohammed A. Al-masni. EMG gesture signal analysis towards diagnosis of upper limb using dual-pathway convolutional neural network[J]. Mathematical Biosciences and Engineering, 2024, 21(4): 5712-5734. doi: 10.3934/mbe.2024252

Related Papers:

Abstract

This research introduces a novel dual-pathway convolutional neural network (DP-CNN) architecture tailored for robust performance in Log-Mel spectrogram image analysis derived from raw multichannel electromyography signals. The primary objective is to assess the effectiveness of the proposed DP-CNN architecture across three datasets (NinaPro DB1, DB2, and DB3), encompassing both able-bodied and amputee subjects. Performance metrics, including accuracy, precision, recall, and F1-score, are employed for comprehensive evaluation. The DP-CNN demonstrates notable mean accuracies of 94.93 ± 1.71% and 94.00 ± 3.65% on NinaPro DB1 and DB2 for healthy subjects, respectively. Additionally, it achieves a robust mean classification accuracy of 85.36 ± 0.82% on amputee subjects in DB3, affirming its efficacy. Comparative analysis with previous methodologies on the same datasets reveals substantial improvements of 28.33%, 26.92%, and 39.09% over the baseline for DB1, DB2, and DB3, respectively. The DP-CNN's superior performance extends to comparisons with transfer learning models for image classification, reaffirming its efficacy. Across diverse datasets involving both able-bodied and amputee subjects, the DP-CNN exhibits enhanced capabilities, holding promise for advancing myoelectric control.

References

[1]	Q. Liu, A. Liu, X. Zhang, X. Chen, R. Qian, X. Chen, Removal of EMG artifacts from multichannel EEG signals using combined singular spectrum analysis and canonical correlation analysis, J. Healthcare Eng., 2019 (2019), 4159676. https://doi.org/10.1155/2019/4159676 doi: 10.1155/2019/4159676
[2]	Y. Kim, S. Stapornchaisit, M. Miyakoshi, N. Yoshimura, Y. Koike, The effect of ICA and non-negative matrix factorization analysis for EMG signals recorded from multi-channel EMG sensors, Front. Neurosci., 14 (2020), 600804. https://doi.org/10.3389/fnins.2020.600804 doi: 10.3389/fnins.2020.600804
[3]	X. Xi, C. Yang, J. Shi, Z. Luo, Y. B. Zhao, Surface electromyography-based daily activity recognition using wavelet coherence coefficient and support vector machine, Neural Process. Lett., 50 (2019), 2265–2280. https://doi.org/10.1007/s11063-019-10008-w doi: 10.1007/s11063-019-10008-w
[4]	Z. Qin, Z. Jiang, J. Chen, C. Hu, Y. Ma, sEMG-based tremor severity evaluation for Parkinson's disease using a light-weight CNN, IEEE Signal Process. Lett., 26 (2019), 637–641. https://doi.org/10.1109/LSP.2019.2903334 doi: 10.1109/LSP.2019.2903334
[5]	K. Leerskov, M. Rehman, I. Niazi, S. Cremoux, M. Jochumsen, Investigating the feasibility of combining EEG and EMG for controlling a hybrid human computer interface in patients with spinal cord injury, in 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE), IEEE, (2020), 403–410. https://doi.org/10.1109/BIBE50027.2020.00072
[6]	F. Amin, A. Waris, J. Iqbal, S. O. Gilani, M. Z. ur Rehman, S. Mushtaq, et al., Maximizing stroke recovery with advanced technologies: A comprehensive assessment of robot-assisted, EMG-Controlled robotics, virtual reality, and mirror therapy interventions, Results Eng., 21 (2024), 101725. https://doi.org/10.1016/j.rineng.2023.101725 doi: 10.1016/j.rineng.2023.101725
[7]	T. W. Boonstra, L. Faes, J. N. Kerkman, D. Marinazzo, Information decomposition of multichannel EMG to map functional interactions in the distributed motor system, NeuroImage, 202 (2019), 116093. https://doi.org/10.1016/j.neuroimage.2019.116093 doi: 10.1016/j.neuroimage.2019.116093
[8]	L. C. Chen, P. H. Chen, R. T. H. Tsai, Y. Tsao, Epg2s: Speech generation and speech enhancement based on electropalatography and audio signals using multimodal learning, IEEE Signal Process. Lett., 29 (2022), 2582–2586. https://doi.org/10.1109/LSP.2022.3184636 doi: 10.1109/LSP.2022.3184636
[9]	S. Inam, S. Al-Harmain, S. Shafique, M. Afzal, A. Rabail, F. Amin, et al., A brief review of strategies used for EMG signal classification, in 2021 International Conference on Artificial Intelligence (ICAI), IEEE, (2021), 140–145. https://doi.org/10.1109/ICAI52203.2021.9445257
[10]	L. Cai, S. Yan, C. Ouyang, T. Zhang, J. Zhu, L. Chen, et al., Muscle synergies in joystick manipulation, Front. Physiol., 14 (2023), 1282295. https://doi.org/10.3389/fphys.2023.1282295 doi: 10.3389/fphys.2023.1282295
[11]	C. Shen, K. Zhang, J. Tang, A COVID-19 detection algorithm using deep features and discrete social learning particle swarm optimization for edge computing devices, ACM Trans. Internet Technol., 22 (2022), 1–17. https://doi.org/10.1145/3453170 doi: 10.1145/3453170
[12]	S. Khalil, U. Nawaz, Zubariah, Z. Mushtaq, S. Arif, M. Z. ur Rehman, et al., Enhancing ductal carcinoma classification using transfer learning with 3D U-Net models in breast cancer imaging, Appl. Sci., 13 (2023), 4255. https://doi.org/10.3390/app13074255 doi: 10.3390/app13074255
[13]	Z. Mushtaq, M. F. Qureshi, M. J. Abbass, S. M. Q. Al-Fakih, Effective kernel-principal component analysis based approach for wisconsin breast cancer diagnosis, Electron. Lett., 59 (2023), e212706. https://doi.org/10.1049/ell2.12706 doi: 10.1049/ell2.12706
[14]	Z. Hu, J. Tang, P. Zhang, J. Jiang, Deep learning for the identification of bruised apples by fusing 3D deep features for apple grading systems, Mech. Syst. Signal Process., 145 (2020), 106922. https://doi.org/10.1016/j.ymssp.2020.106922 doi: 10.1016/j.ymssp.2020.106922
[15]	A. Shahzad, A. Mushtaq, A. Q. Sabeeh, Y. Y. Ghadi, Z. Mushtaq, S. Arif, et al., Automated uterine fibroids detection in ultrasound images using deep convolutional neural networks, Healthcare, 11 (2023), 1493. https://doi.org/10.3390/healthcare11101493 doi: 10.3390/healthcare11101493
[16]	N. Afshan, Z. Mushtaq, F. S. Alamri, M. F. Qureshi, N. A. Khan, I. Siddique, Efficient thyroid disorder identification with weighted voting ensemble of super learners by using adaptive synthetic sampling technique, AIMS Math., 8 (2023), 24274–24309. https://doi.org/10.3934/math.20231238 doi: 10.3934/math.20231238
[17]	A. A. Khan, S. Raza, M. F. Qureshi, Z. Mushtaq, M. Taha, F. Amin, Deep learning-based classification of wheat leaf diseases for edge devices, in 2023 2nd International Conference on Emerging Trends in Electrical, Control, and Telecommunication Engineering (ETECTE), IEEE, (2023), 1–6. https://doi.org/10.1109/ETECTE59617.2023.10396676
[18]	D. Huang, B. Chen, Surface EMG decoding for hand gestures based on spectrogram and CNN-LSTM, in 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI), IEEE, (2019), 123–126. https://doi.org/10.1109/CCHI.2019.8901936
[19]	J. O. Pinzón-Arenas, R. Jiménez-Moreno, A. Rubiano, Percentage estimation of muscular activity of the forearm by means of EMG signals based on the gesture recognized using CNN, Sens. Bio-Sens. Res., 29 (2020), 100353. https://doi.org/10.1016/j.sbsr.2020.100353 doi: 10.1016/j.sbsr.2020.100353
[20]	B. Saeed, S. O. Gilani, Z. ur Rehman, M. Jamil, A. Waris, M. N. Khan, Comparative analysis of classifiers for EMG signals, in 2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE), IEEE, (2019), 1–5. https://doi.org/10.1109/CCECE.2019.8861835
[21]	N. K. Karnam, A. C. Turlapaty, S. R. Dubey, B. Gokaraju, Classification of sEMG signals of hand gestures based on energy features, Biomed. Signal Process. Control, 70 (2021), 102948. https://doi.org/10.1016/j.bspc.2021.102948 doi: 10.1016/j.bspc.2021.102948
[22]	M. Akmal, S. Khalid, M. Moiz, M. J. Abbass, M. F. Qureshi, Z. Mushtaq, Leveraging training strategies of artificial neural network for classification of multiday electromyography signals, in 2022 International Conference on Emerging Trends in Electrical, Control, and Telecommunication Engineering (ETECTE), IEEE, (2022), 1–5. https://doi.org/10.1109/ETECTE55893.2022.10007103
[23]	M. Akmal, M. F. Qureshi, F. Amin, M. Z. ur Rehman, I. K. Niazi, SVM-based real-time classification of prosthetic fingers using myo armband-acquired electromyography data, in 2021 IEEE 21st International Conference on Bioinformatics and Bioengineering (BIBE), IEEE, (2021), 1–5. https://doi.org/10.1109/BIBE52308.2021.9635461
[24]	S. Inam, F. Amin, M. Z. ur Rehman, Comparative study of flexor and extensor muscles emg for upper limb prosthesis, in 2021 15th International Conference on Open Source Systems and Technologies (ICOSST), IEEE, (2021), 1–5. https://doi.org/10.1109/ICOSST53930.2021.9683956
[25]	Y. Hu, Y. Wong, W. Wei, Y. Du, M. Kankanhalli, W. Geng, A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition, PLoS One, 13 (2018), e0206049. https://doi.org/10.1371/journal.pone.0206049 doi: 10.1371/journal.pone.0206049
[26]	S. Pancholi, A. M. Joshi, D. Joshi, A robust and accurate deep learning based pattern recognition framework for upper limb prosthesis using semg, preprint, arXiv: 2106.02463.
[27]	Y. Cheng, G. Li, M. Yu, D. Jiang, J. Yun, Y. Liu, et al., Gesture recognition based on surface electromyography-feature image, Concurrency Comput. Pract. Exper., 33 (2021), e6051. https://doi.org/10.1002/cpe.6051 doi: 10.1002/cpe.6051
[28]	R. Tong, Y. Zhang, H. Chen, H. Liu, Learn the temporal-spatial feature of sEMG via dual-flow network, Int. J. Humanoid Rob., 16 (2019), 1941004. https://doi.org/10.1142/S0219843619410044 doi: 10.1142/S0219843619410044
[29]	P. Xu, F. Li, H. Wang, A novel concatenate feature fusion RCNN architecture for sEMG-based hand gesture recognition, PLoS One, 17 (2022), e0262810. https://doi.org/10.1371/journal.pone.0262810 doi: 10.1371/journal.pone.0262810
[30]	M. F. Qureshi, Z. Mushtaq, M. Z. ur Rehman, E. N. Kamavuako, E2cnn: An efficient concatenated cnn for classification of surface emg extracted from upper limb, IEEE Sens. J., 23 (2023), 8989–8996. https://doi.org/10.1109/JSEN.2023.3255408 doi: 10.1109/JSEN.2023.3255408
[31]	M. F. Qureshi, Z. Mushtaq, M. Z. ur Rehman, E. N. Kamavuako, Spectral image-based multiday surface electromyography classification of hand motions using CNN for human–computer interaction, IEEE Sens. J., 22 (2022), 20676–20683. https://doi.org/10.1109/JSEN.2022.3204121 doi: 10.1109/JSEN.2022.3204121
[32]	H. Nodera, Y. Osaki, H. Yamazaki, A. Mori, Y. Izumi, R. Kaji, Deep learning for waveform identification of resting needle electromyography signals, Clin. Neurophysiol., 130 (2019), 617–623. https://doi.org/10.1016/j.clinph.2019.01.024 doi: 10.1016/j.clinph.2019.01.024
[33]	D. Gao, X. Tang, M. Wan, G. Huang, Y. Zhang, EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks, Front. Neurosci., 17 (2023), 1136609. https://doi.org/10.3389/fnins.2023.1136609 doi: 10.3389/fnins.2023.1136609
[34]	T. Tuncer, S. Dogan, M. Baygin, U. R. Acharya, Tetromino pattern based accurate eeg emotion classification model, Artif. Intell. Med., 123 (2022), 102210. https://doi.org/10.1016/j.artmed.2021.102210 doi: 10.1016/j.artmed.2021.102210
[35]	M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A. G. M. Hager, S. Elsig, et al., Electromyography data for non-invasive naturally-controlled robotic hand prostheses, Sci. Data, 1 (2014), 140053. https://doi.org/10.1038/sdata.2014.53 doi: 10.1038/sdata.2014.53
[36]	A. Gijsberts, M. Atzori, C. Castellini, H. Müller, B. Caputo, Measuring movement classification performance with the movement error rate, IEEE Trans. Neural Syst. Rehabil. Eng., 89621 (2014), 735–744.
[37]	M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A. G. M. Hager, E. Simone, et al., Clinical parameter effect on the capability to control myoelectric robotic prosthetic hands, J. Rehabil. Res. Dev., 53 (2016), 345–358. http://doi.org/10.1682/JRRD.2014.09.0218 doi: 10.1682/JRRD.2014.09.0218
[38]	M. Atzori, M. Cognolato, H. Müller, Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands, Front. Neurorob., 10 (2016), 9. https://doi.org/10.3389/fnbot.2016.00009 doi: 10.3389/fnbot.2016.00009
[39]	X. Zhang, X. Li, O. W. Samuel, Z. Huang, P. Fang, G. Li, Improving the robustness of electromyogram-pattern recognition for prosthetic control by a postprocessing strategy, Front. Neurorob., 11 (2017), 51. https://doi.org/10.3389/fnbot.2017.00051 doi: 10.3389/fnbot.2017.00051
[40]	F. Riillo, L. Quitadamo, F. Cavrini, E. Gruppioni, C. Pinto, N. C. Pastò, et al., Optimization of EMG-based hand gesture recognition: Supervised vs. unsupervised data preprocessing on healthy subjects and transradial amputees, Biomed. Signal Process. Control, 14 (2014), 117–125. https://doi.org/10.1016/j.bspc.2014.07.007 doi: 10.1016/j.bspc.2014.07.007
[41]	S. Lu, J. Yang, B. Yang, X. Li, Z. Yin, L. Yin, et al., Surgical instrument posture estimation and tracking based on lstm, ICT Express, in press. https://doi.org/10.1016/j.icte.2024.01.002
[42]	S. Zhao, W. Liang, K. Wang, L. Ren, Z. Qian, G. Chen, et al., A multiaxial bionic ankle based on series elastic actuation with a parallel spring, IEEE Trans. Ind. Electron., 2023 (2023), 1–13. https://doi.org/10.1109/TIE.2023.3310041 doi: 10.1109/TIE.2023.3310041
[43]	W. Geng, Y. Du, W. Jin, W. Wei, Y. Hu, J. Li, Gesture recognition by instantaneous surface EMG images, Sci. Rep., 6 (2016), 36571. https://doi.org/10.1038/srep36571 doi: 10.1038/srep36571
[44]	W. Wei, Y. Wong, Y. Du, Y. Hu, M. Kankanhalli, W. Geng, A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface, Pattern Recognit. Lett., 119 (2019), 131–138. https://doi.org/10.1016/j.patrec.2017.12.005 doi: 10.1016/j.patrec.2017.12.005
[45]	W. Wei, Q. Dai, Y. Wong, Y. Hu, M. Kankanhalli, W. Geng, Surface-electromyography-based gesture recognition by multi-view deep learning, IEEE Trans. Biomed. Eng., 66 (2019), 2964–2973. https://doi.org/10.1109/TBME.2019.2899222 doi: 10.1109/TBME.2019.2899222
[46]	H. Wang, Y. Zhang, C. Liu, H. Liu, sEMG based hand gesture recognition with deformable convolutional network, Int. J. Mach. Learn. Cybern., 13 (2022), 1729–1738. https://doi.org/10.1007/s13042-021-01482-7 doi: 10.1007/s13042-021-01482-7
[47]	Y. Zhang, F. Yang, Q. Fan, A. Yang, X. Li, Research on sEMG-based gesture recognition by dual-view deep learning, IEEE Access, 10 (2022), 32928–32937. https://doi.org/10.1109/ACCESS.2022.3158667 doi: 10.1109/ACCESS.2022.3158667
[48]	X. Zhai, B. Jelfs, R. H. M. Chan, C. Tin, Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network, Front. Neurorob., 11 (2017), 379. https://doi.org/10.3389/fnins.2017.00379 doi: 10.3389/fnins.2017.00379
[49]	J. A. Sandoval-Espino, A. Zamudio-Lara, J. A. Marbán-Salgado, J. J. Escobedo-Alatorre, O. Palillero-Sandoval, J. G. Velásquez-Aguilar, Selection of the best set of features for sEMG-based hand gesture recognition applying a CNN architecture, Sensors, 22 (2022), 4972. https://doi.org/10.3390/s22134972 doi: 10.3390/s22134972

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)