Research on two-class and four-class action recognition based on EEG signals

Ying Chang; Lan Wang; Yunmin Zhao; Ming Liu; Jing Zhang; Ying Chang; Lan Wang; Yunmin Zhao; Ming Liu; Jing Zhang

doi:10.3934/mbe.2023455

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 6: 10376-10391. doi: 10.3934/mbe.2023455

Previous Article Next Article

Research article Special Issues

Research on two-class and four-class action recognition based on EEG signals

1.
College of Mechanical and Electrical Engineering, Harbin Engineering University, Harbin 150006, China
2.
School of Mechanical and Civil Engineering, Jilin Agricultural Science and Technology University, Jilin 132109, China
3.
College of Electronic and Information Engineering, Tongji University, Shanghai 200092, China
4.
Technology Department YAMAMOTO CO., LTD, Higashine-shi 999-3701, Japan
5.
Respiratory Department, JiLin Central Hospital, Jilin 132109, China

Academic Editor: Simon James Fong

Received: 14 February 2023 Revised: 09 March 2023 Accepted: 19 March 2023 Published: 06 April 2023

BMI has attracted widespread attention in the past decade, which has greatly improved the living conditions of patients with motor disorders. The application of EEG signals in lower limb rehabilitation robots and human exoskeleton has also been gradually applied by researchers. Therefore, the recognition of EEG signals is of great significance. In this paper, a CNN-LSTM neural network model is designed to study the two-class and four-class motion recognition of EEG signals. In this paper, a brain-computer interface experimental scheme is designed. Combining the characteristics of EEG signals, the time-frequency characteristics of EEG signals and event-related potential phenomena are analyzed, and the ERD/ERS characteristics are obtained. Pre-process EEG signals, and propose a CNN-LSTM neural network model to classify the collected binary and four-class EEG signals. The experimental results show that the CNN-LSTM neural network model has a good effect, and its average accuracy and kappa coefficient are higher than the other two classification algorithms, which also shows that the classification algorithm selected in this paper has a good classification effect.
- EEG,
- CNN-LSTM,
- motion recognition,
- data acquisition
Citation: Ying Chang, Lan Wang, Yunmin Zhao, Ming Liu, Jing Zhang. Research on two-class and four-class action recognition based on EEG signals[J]. Mathematical Biosciences and Engineering, 2023, 20(6): 10376-10391. doi: 10.3934/mbe.2023455

Related Papers:

Abstract

BMI has attracted widespread attention in the past decade, which has greatly improved the living conditions of patients with motor disorders. The application of EEG signals in lower limb rehabilitation robots and human exoskeleton has also been gradually applied by researchers. Therefore, the recognition of EEG signals is of great significance. In this paper, a CNN-LSTM neural network model is designed to study the two-class and four-class motion recognition of EEG signals. In this paper, a brain-computer interface experimental scheme is designed. Combining the characteristics of EEG signals, the time-frequency characteristics of EEG signals and event-related potential phenomena are analyzed, and the ERD/ERS characteristics are obtained. Pre-process EEG signals, and propose a CNN-LSTM neural network model to classify the collected binary and four-class EEG signals. The experimental results show that the CNN-LSTM neural network model has a good effect, and its average accuracy and kappa coefficient are higher than the other two classification algorithms, which also shows that the classification algorithm selected in this paper has a good classification effect.

References

[1]	A. Venkatakrishnan, G. E. Francisco, J. L. Contreras-Vidal, Applications of brain–machine interface systems in stroke recovery and rehabilitation, Curr. Phys. Med. Rehabil. Rep., 2 (2014), 93–105. https://doi.org/10.1007/s40141-014-0051-4 doi: 10.1007/s40141-014-0051-4
[2]	I. K. Niazi, N. Jiang, M. Jochumsen, J. F. Nielsen, K. Dremstrup, D. Farina, Detection of movement re1ated cortical potentials based on subject independent training, Med. Biol. Eng. Comput., 51 (2013), 507–512. https://doi.org/10.1007/s11517-012-1018-1 doi: 10.1007/s11517-012-1018-1
[3]	A. Presacco, L. W. Forrester, J. L. Contreras-Vidal, Decoding intra-limb and inter-limb kinematics during treadmill walking from scalp electroencephalographic (EEG) signals, IEEE Trans. Neural Syst. Rehabil. Eng., 20 (2012), 212–219. https://doi.org/10.1109/TNSRE.2012.2188304 doi: 10.1109/TNSRE.2012.2188304
[4]	N. A. Fitzsimmons, M. A. Lebedev, I. D. Peikon, M. A. L. Nicolelis, Extracting kinematic parameters for monkey bipedal walking from cortical neuronal ensemble activity, Front. Integr. Neurosci., 3 (2009), 3. https://doi.org/10.3389/neuro.07.003.2009 doi: 10.3389/neuro.07.003.2009
[5]	J. L. Contreras-Vidal, A. Kilicarslan, H. Huang, R. G. Grossman, Human-Centered design of wearable neuroprostheses and exoskeletons, Ai Mag. Artif. Intell., 36 (2015), 12–22. https://doi.org/10.1609/aimag.v36i4.2613 doi: 10.1609/aimag.v36i4.2613
[6]	Nidhi, D. Joshi, Terrain-based gait recognition using EEG: Comparing machine learning and deep learning models, in 2021 International Conference on Computational Performance Evaluation (ComPE), (2021), 734–740. https://doi.org/10.1109/ComPE53109.2021.9751957
[7]	S. M. S. Hasan, M. R. Siddiquee, R. Atri, R. Ramon, J. S. Marquez, O. Bai, Prediction of gait intention from pre-movement EEG signals: A feasibility study, J. NeuroEng. Rehabil., 17 (2020), 50. https://doi.org/10.1186/s12984-020-00675-5 doi: 10.1186/s12984-020-00675-5
[8]	I. Walker, Deep convolutional neural networks for brain computer interface using motor imaginary, Master thesis, Imperial College London, 2015.
[9]	L. Clemente, L. Garrido, EEG binary classification using convolutional neural networks, Campus Monterrey, 2016.
[10]	Z. C. Tang, K. J. Zhang, C. Li, Classification of motor imagery based on deep convolution neural network and its application in brain-controlled exoskeleton by EEG, J. Comput. Sci., 40 (2017), 12. https://doi.org/10.11897/SP.J.1016.2017.01367 doi: 10.11897/SP.J.1016.2017.01367
[11]	D. J. Leamy, J. Kocijan, K. Domijan, J. Duffin, R. A. Roche, S. Commins, et al., An exploration of EEG features during recovery following stroke-implications for BCI-mediated neurorehabilitation therapy, J. Neuroeng. Rehabil., 11 (2014), 9. https://doi.org/10.1186/1743-0003-11-9 doi: 10.1186/1743-0003-11-9
[12]	E. Hortal, D. Planelles, A. Costa, E. Iáñez, A. Úbeda, J.M. Azorín, et al., SVM-based Brain–Machine Interface for controlling a robot arm through four mental tasks, Neurocomputing, 151 (2015), 116–121. https://doi.org/10.1016/j.neucom.2014.09.078 doi: 10.1016/j.neucom.2014.09.078
[13]	N. Jiang, L. Gizzi, N. Mrachacz-Kersting, K. Dremstrup, D. Farina, A brain-computer interface for single-trial detection of gait initiation from movement related cortical potentials, Clin. Neurophys., 126 (2015), 154–159. https://doi.org/10.1016/j.clinph.2014.05.003 doi: 10.1016/j.clinph.2014.05.003
[14]	Z. Jiang, P. Liu, Y. Xia, J. Zhang, Application of CNN in EEG image classification of AD patients, in The 2nd International Conference on Computing and Data Science, 21 (2021), 1–5. https://doi.org/10.1145/3448734.3450473
[15]	D. Garrett, D. A. Peterson, C. W. Anderson, M. H. Thaut, Comparison of linear, nonlinear, and feature selection methods for EEG signal classification, IEEE Trans. Neural Syst. Rehabil. Eng., 11 (2003), 141–144. https://doi.org/10.1109/TNSRE.2003.814441 doi: 10.1109/TNSRE.2003.814441
[16]	T. N. Lal, M. Schröder, T. Hinterberger, J. Weston, M. Bogdan, N. Birbaumer, et al., Support vector channel selection in BCI, IEEE Trans. Biomed. Eng., 51 (2004), 1003–1010. https://doi.org/10.1109/TBME.2004.827827 doi: 10.1109/TBME.2004.827827
[17]	F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, B. Arnaldi, A review of classification algorithms for EEG-based brain–computer interfaces, J. Neural Eng., 4 (2007). https://doi.org/10.1088/1741-2560/4/2/R01
[18]	S. Sonnenburg, G. Rätsch, C. Schäfer, B. Schölkopf, Large scale multiple kernel learning, J. Mach. Learn. Res., 7 (2006), 1531–1565. https://doi.org/10.1007/s10450-006-0008-8 doi: 10.1007/s10450-006-0008-8
[19]	X. Li, X. Chen, Y. Yan, W. Wei, Z. J. Wang, Classification of EEG signals using a multiple kernel learning support vector machine, Sensors, 14 (2014), 12784–12802. https://doi.org/10.3390/s140712784 doi: 10.3390/s140712784
[20]	Y. Zhang, S. Prasad, A. Kilicarslan, J. L. Contreras-Vidal, Multiple kernel based region importance learning for neural classification of gait states from EEG signals, Front. Neurosci., 11 (2017), 170. https://doi.org/10.3389/fnins.2017.00170 doi: 10.3389/fnins.2017.00170
[21]	Neeraj, V. Singhal, J. Mathew, R. K. Behera, Detection of alcoholism using EEG signals and a CNN-LSTM-ATTN network, Comput. Biol. Med., 138 (2021), 104940. https://doi.org/10.1016/j.compbiomed.2021.104940 doi: 10.1016/j.compbiomed.2021.104940
[22]	C. Chen, Z. Du, L. He, Y. Shi, J. Wang, W. Dong, A novel gait pattern recognition method based on LSTM-CNN for lower limb exoskeleton, J. Bionic Eng., 18 (2021), 1059–1072. https://doi.org/10.1007/s42235-021-00083-y doi: 10.1007/s42235-021-00083-y
[23]	L. F. Shi, Z. Y. Liu, K. J. Zhou, Y. Shi, X. Jing, Novel deep learning network for gait recognition using multimodal inertial sensors, Sensors (Basel), 23 (2023), 849. https://doi.org/10.3390/s23020849 doi: 10.3390/s23020849
[24]	J. Gao, P. Gu, Q. Ren, J. Zhang, X. Song, Abnormal gait recognition algorithm based on LSTM-CNN fusion network, IEEE Access, 7 (2019), 163180–163190. https://doi.org/10.1109/ACCESS.2019.2950254. doi: 10.1109/ACCESS.2019.2950254
[25]	G. Pfurtscheller, F. H. L. da Silva, Event-related EEG/MEG synchronization and desynchronization, Clin. Neurophysiol., 110 (1999), 1842–1857. https://doi.org/10.1016/s1388-2457(99)00141-8 doi: 10.1016/s1388-2457(99)00141-8
[26]	Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature, 521 (2015), 436–444. https://doi.org/10.1142/S1793351X16500045 doi: 10.1142/S1793351X16500045
[27]	D. Lin, F. Duan, W. Li, J. Shen, Q. M. Wang, X. Luo, Optimizing the individual differences of EEG signals through BP neural network algorithm for a BCI dialing system, in International Conference on Brain & Health Informatics, (2013), 479–488. https://doi.org/10.1007/978-3-319-02753-1_48
[28]	X. An, D. Kuang, X. Guo, Y. Zhao, L. He, A deep learning method for classification of EEG data based on motor imagery, in International Conference on Intelligent Computing, (2014), 203–210. https://doi.org/10.1007/978-3-319-09330-7_25
[29]	S. U. Amin, M. Alsulaiman, G. Muhammad, M. A. Mekhtiche, M. S. Hossain, Deeplearning for EEG motor imagery classification based on multi-layer CNNs feature fusion, Future Gener. Comput. Syst., 101 (2019), 542–554. https://doi.org/10.1016/j.future.2019.06.027 doi: 10.1016/j.future.2019.06.027
[30]	N. Nazari, S. A. Mirsalari, S. Sinaei, M. E. Salehi, M. Daneshtalab, Multi-level binarized LSTM in EEG classification for wearable devices, (2020), 175–181. https://doi.org/10.1109/PDP50117.2020.00033

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)