A multi-feature fusion decoding study for unilateral upper-limb fine motor imagery

Liangyu Yang; Tianyu Shi; Jidong Lv; Yan Liu; Yakang Dai; Ling Zou; Liangyu Yang; Tianyu Shi; Jidong Lv; Yan Liu; Yakang Dai; Ling Zou

doi:10.3934/mbe.2023116

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 2: 2482-2500. doi: 10.3934/mbe.2023116

Previous Article Next Article

Research article

A multi-feature fusion decoding study for unilateral upper-limb fine motor imagery

1.
The School of Microelectronics and Control Engineering, Changzhou University, Changzhou, Jiangsu 213164, China
2.
Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Department of Medical Image, Suzhou 215163, China
3.
Suzhou Guokekangcheng Medical Technique Co., Ltd., Suzhou 215163, China
4.
Key Laboratory of Brain Machine Collaborative Intelligence Foundation of Zhejiang Province, Hangzhou, Zhejiang 310018, China

Academic Editor: Yang Kuang

Received: 28 September 2022 Revised: 05 November 2022 Accepted: 11 November 2022 Published: 22 November 2022

To address the fact that the classical motor imagination paradigm has no noticeable effect on the rehabilitation training of upper limbs in patients after stroke and the corresponding feature extraction algorithm is limited to a single domain, this paper describes the design of a unilateral upper-limb fine motor imagination paradigm and the collection of data from 20 healthy people. It presents a feature extraction algorithm for multi-domain fusion and compares the common spatial pattern (CSP), improved multiscale permutation entropy (IMPE) and multi-domain fusion features of all participants through the use of decision tree, linear discriminant analysis, naive Bayes, a support vector machine, k-nearest neighbor and ensemble classification precision algorithms in the ensemble classifier. For the same subject, the average classification accuracy improvement of the same classifier for multi-domain feature extraction relative to CSP feature results went up by 1.52%. The average classification accuracy improvement of the same classifier went up by 32.87% relative to the IMPE feature classification results. This study's unilateral fine motor imagery paradigm and multi-domain feature fusion algorithm provide new ideas for upper limb rehabilitation after stroke.
- brain-computer interface,
- upper limb rehabilitation,
- unilateral fine motor imagery,
- multi-domain fusion
Citation: Liangyu Yang, Tianyu Shi, Jidong Lv, Yan Liu, Yakang Dai, Ling Zou. A multi-feature fusion decoding study for unilateral upper-limb fine motor imagery[J]. Mathematical Biosciences and Engineering, 2023, 20(2): 2482-2500. doi: 10.3934/mbe.2023116

Related Papers:

Abstract

To address the fact that the classical motor imagination paradigm has no noticeable effect on the rehabilitation training of upper limbs in patients after stroke and the corresponding feature extraction algorithm is limited to a single domain, this paper describes the design of a unilateral upper-limb fine motor imagination paradigm and the collection of data from 20 healthy people. It presents a feature extraction algorithm for multi-domain fusion and compares the common spatial pattern (CSP), improved multiscale permutation entropy (IMPE) and multi-domain fusion features of all participants through the use of decision tree, linear discriminant analysis, naive Bayes, a support vector machine, k-nearest neighbor and ensemble classification precision algorithms in the ensemble classifier. For the same subject, the average classification accuracy improvement of the same classifier for multi-domain feature extraction relative to CSP feature results went up by 1.52%. The average classification accuracy improvement of the same classifier went up by 32.87% relative to the IMPE feature classification results. This study's unilateral fine motor imagery paradigm and multi-domain feature fusion algorithm provide new ideas for upper limb rehabilitation after stroke.

References

[1]	S. Aggarwal, N. Chugh, Signal processing techniques for motor imagery brain computer interface: a review, Array, 1–2 (2019), 100003. https://doi.org/10.1016/j.array.2019.100003 doi: 10.1016/j.array.2019.100003
[2]	B. Yang, J. Ma, W. Qiu, Y. Zhu, X. Meng, A new 2-class unilateral upper limb motor imagery tasks for stroke rehabilitation training, Med. Novel Technol. Devices, 13 (2022), 100100. https://doi.org/10.1016/j.medntd.2021.100100 doi: 10.1016/j.medntd.2021.100100
[3]	M. A. Cervera, S. R. Soekadar, J. Ushiba, J. del R. Millán, M. Liu, N. Birbaumer, et al., Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis, Ann. Clin. Transl. Neurol. , 5 (2018), 651–663. https://doi.org/10.1002/acn3.544 doi: 10.1002/acn3.544
[4]	U. Chaudhary, N. Birbaumer, A. Ramos-Murguialday, Brain-computer interfaces for communication and rehabilitation, Nat. Rev. Neurol. , 12 (2016), 513–525. https://doi.org/10.1038/nrneurol.2016.113 doi: 10.1038/nrneurol.2016.113
[5]	R. Mane, T. Chouhan, C. Guan, BCI for stroke rehabilitation: motor and beyond, J. Neural Eng. , 17 (2020), 041001. https://doi.org/10.1088/1741-2552/aba162 doi: 10.1088/1741-2552/aba162
[6]	H. Dose, J. S. Møller, H. K. Iversen, S. Puthusserypady, An end-to-end deep learning approach to MI-EEG signal classification for BCIs, Expert Syst. Appl. , 114 (2018), 532–542. https://doi.org/10.1016/j.eswa.2018.08.031 doi: 10.1016/j.eswa.2018.08.031
[7]	Y. Zhang, W. Chen, C. L. Lin, Z. Pei, J. Chen, Z. Chen, Boosting-LDA algriothm with multi-domain feature fusion for motor imagery EEG decoding, Biomed. Signal Process. Control, 70 (2021), 102983. https://doi.org/10.1016/j.bspc.2021.102983 doi: 10.1016/j.bspc.2021.102983
[8]	A. Khalaf, E. Sejdic, M. Akcakaya, Common spatial pattern and wavelet decomposition for motor imagery EEG- fTCD brain-computer interface, J. Neurosci. Methods, 320 (2019), 98–106. https://doi.org/10.1016/j.jneumeth.2019.03.018 doi: 10.1016/j.jneumeth.2019.03.018
[9]	P. Gaur, H. Gupta, A. Chowdhury, K. McCreadie, R. B. Pachori, H. Wang, A sliding window common spatial pattern for enhancing motor imagery classification in EEG-BCI, IEEE Trans. Instrum. Meas. , 70 (2021), 1–9. https://doi.org/10.1109/TIM.2021.3051996 doi: 10.1109/TIM.2021.3051996
[10]	Y. Zhang, C. S. Nam, G. Zhou, J. Jin, X. Wang, A. Cichocki, Temporally constrained sparse group spatial patterns for motor imagery BCI, IEEE Trans. Cybern. , 49 (2019) 3322–3332. https://doi.org/10.1109/TCYB.2018.2841847 doi: 10.1109/TCYB.2018.2841847
[11]	N. S. Malan, S. Sharma, Motor imagery EEG spectral-spatial feature optimization using dual-tree complex wavelet and neighbourhood component analysis, IRBM, 43 (2022), 198–209. https://doi.org/10.1016/j.irbm.2021.01.002 doi: 10.1016/j.irbm.2021.01.002
[12]	K. K. Ang, Z. Y. Chin, H. Zhang, C. Guan, Filter Bank Common Spatial Pattern (FBCSP) in brain-computer interface, in 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), (2008), 2390–2397. https://doi.org/10.1109/IJCNN.2008.4634130
[13]	K. Sivasankari, K. Thanushkodi, An improved EEG signal classification using neural network with the consequence of ICA and STFT, J. Electr. Eng. Technol. , 9 (2014), 1060–1071. https://doi.org/10.5370/JEET.2014.9.3.1060 doi: 10.5370/JEET.2014.9.3.1060
[14]	M. Diykh, Y. Li, P. Wen, EEG sleep stages classification based on time domain features and structural graph similarity, IEEE Trans. Neural Syst. Rehabil. Eng. , 24 (2016), 1159–1168. https://doi.org/10.1109/TNSRE.2016.2552539 doi: 10.1109/TNSRE.2016.2552539
[15]	N. K. Al-Qazzaz, M. K. Sabir, S. H. B. M. Ali, S. A. Ahmad, K. Grammer, Multichannel optimization with hybrid spectral- entropy markers for gender identification enhancement of emotional-based EEGs, IEEE Access, 9 (2021), 107059–107078. https://doi.org/10.1109/ACCESS.2021.3096430 doi: 10.1109/ACCESS.2021.3096430
[16]	N. Mammone, F. La Foresta, F. C. Morabito, Automatic artifact rejection from multichannel scalp EEG by wavelet ICA, IEEE Sens. J. , 12 (2012), 533–542. https://doi.org/10.1109/JSEN.2011.2115236 doi: 10.1109/JSEN.2011.2115236
[17]	X. Liu, G. Wang, J. Gao, Q. Gao, A quantitative analysis for EEG signals based on modified permutation-entropy, IRBM, 38 (2017), 71–77. https://doi.org/10.1016/j.irbm.2017.02.001 doi: 10.1016/j.irbm.2017.02.001
[18]	D. Q. Phung, D. Tran, W. Ma, P. Nguyen, T. Pham, Using shannon entropy as EEG signal feature for fast person identification, ESANN, 4 (2014), 413–418.
[19]	X. Jie, R. Cao, L. Li, Emotion recognition based on the sample entropy of EEG, Bio-Med. Mater. Eng. , 24 (2014), 1185–1192. https://doi.org/10.3233/BME-130919 doi: 10.3233/BME-130919
[20]	Y. Park, W. Chung, Optimal channel selection using correlation coefficient for CSP based EEG classification, IEEE Access, 8 (2020), 111514–111521. https://doi.org/10.1109/ACCESS.2020.3003056 doi: 10.1109/ACCESS.2020.3003056
[21]	X. Liu, Y. Shen, J. Liu, J. Yang, P. Xiong, F. Lin, Parallel spatial–temporal self-attention CNN-based motor imagery classification for BCI, Front. Neurosci. , 14 (2020), 587520. https://doi.org/10.3389/fnins.2020.587520 doi: 10.3389/fnins.2020.587520
[22]	Y. A. Baysal, S. Ketenci, I. H. Altas, T. Kayikcioglu, Multi-objective symbiotic organism search algorithm for optimal feature selection in brain computer interfaces, Expert Syst. Appl. , 165 (2021), 113907. https://doi.org/10.1016/j.eswa.2020.113907 doi: 10.1016/j.eswa.2020.113907
[23]	Z. Yu, W. Chen, T. Zhang, Motor imagery EEG classification algorithm based on improved lightweight feature fusion network, Biomed. Signal Process. Control, 75 (2022), 103618. https://doi.org/10.1016/j.bspc.2022.103618 doi: 10.1016/j.bspc.2022.103618
[24]	C. Wang, Y. Wu, C. Wang, Y. Zhu, C. Wang, Y. Niu, et al., MI-EEG classification using Shannon complex wavelet and convolutional neural networks, Appl. Soft Comput. , 130 (2022), 109685. https://doi.org/10.1016/j.asoc.2022.109685 doi: 10.1016/j.asoc.2022.109685
[25]	A. Jafarifarmand, M. A. Badamchizadeh, Real-time multiclass motor imagery brain-computer interface by modified common spatial patterns and adaptive neuro-fuzzy classifier, Biomed. Signal Process. Control, 57 (2020), 101749. https://doi.org/10.1016/j.bspc.2019.101749 doi: 10.1016/j.bspc.2019.101749
[26]	X. Zheng, J. Li, H. Ji, L. Duan, M. Li, Z. Pang, et al., Task transfer learning for EEG classification in motor imagery-based BCI system, Comput. Math. Methods Med. , 2020 (2020), e6056383. https://doi.org/10.1155/2020/6056383 doi: 10.1155/2020/6056383
[27]	Y. Xu, Q. Wei, H. Zhang, R. Hu, J. Liu, J. Hua, et al., Transfer learning based on regularized common spatial patterns using cosine similarities of spatial filters for motor-imagery BCI, J. Circuits Syst. Comput. , 28 (2019), 1950123. https://doi.org/10.1142/S0218126619501238 doi: 10.1142/S0218126619501238
[28]	M. Z. Baig, N. Aslam, H. P. H. Shum, L. Zhang, Differential evolution algorithm as a tool for optimal feature subset selection in motor imagery EEG, Expert Syst. Appl. , 90 (2017), 184–195. https://doi.org/10.1016/j.eswa.2017.07.033 doi: 10.1016/j.eswa.2017.07.033
[29]	H. Azami, J. Escudero, Improved multiscale permutation entropy for biomedical signal analysis: interpretation and application to electroencephalogram recordings, Biomed. Signal Process. Control, 23 (2016), 28–41. https://doi.org/10.1016/j.bspc.2015.08.004 doi: 10.1016/j.bspc.2015.08.004
[30]	M. E. S. H. Jomaa, P. Van Bogaert, N. Jrad, N. E. Kadish, N. Japaridze, M. Siniatchkin, et al., Multivariate improved weighted multiscale permutation entropy and its application on EEG data, Biomed. Signal Process. Control, 52 (2019), 420–428. https://doi.org/10.1016/j.bspc.2018.08.004 doi: 10.1016/j.bspc.2018.08.004
[31]	Y. Miao, J. Jin, I. Daly, C. Zuo, X. Wang, A. Cichocki, et al., Learning common time-frequency-spatial patterns for motor imagery classification, IEEE Trans. Neural Syst. Rehabil. Eng. , 29 (2021), 699–707. https://doi.org/10.1109/TNSRE.2021.3071140 doi: 10.1109/TNSRE.2021.3071140
[32]	Y. Hou, T. Chen, X. Lun, F. Wang, A novel method for classification of multi-class motor imagery tasks based on feature fusion, Neurosci. Res. , 176 (2022), 40–48. https://doi.org/10.1016/j.neures.2021.09.002 doi: 10.1016/j.neures.2021.09.002
[33]	L. Hu, J. Xie, C. Pan, X. Wu, D. Hu, Multi-feature fusion method based on WOSF and MSE for four-class MI EEG identification, Biomed. Signal Process. Control, 69 (2021), 102907. https://doi.org/10.1016/j.bspc.2021.102907 doi: 10.1016/j.bspc.2021.102907
[34]	Y. Djenouri, A. Belhadi, A. Yazidi, G. Srivastava, J. C. W. Lin, Artificial intelligence of medical things for disease detection using ensemble deep learning and attention mechanism, Expert Syst., e13093 (2022). https://doi.org/10.1111/exsy.13093
[35]	T. Zhang, W. Chen, M. Li, Classification of inter-ictal and ictal EEGs using multi-basis MODWPT, dimensionality reduction algorithms and LS-SVM: a comparative study, Biomed. Signal Process. Control, 47 (2019), 240–251. https://doi.org/10.1016/j.bspc.2018.08.038
[36]	W. Pan, Y. An, Y. Guan, J. Wang, MCA-net: a multi-task channel attention network for Myocardial infarction detection and location using 12-lead ECGs, Comput. Biol. Med. , 150 (2022), 106199. https://doi.org/10.1016/j.compbiomed.2022.106199 doi: 10.1016/j.compbiomed.2022.106199

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)