Research article Special Issues

Research on gesture recognition algorithm based on MME-P3D


  • A Multiscale-Motion Embedding Pseudo-3D (MME-P3D) gesture recognition algorithm has been proposed to tackle the issues of excessive parameters and high computational complexity encountered by existing gesture recognition algorithms deployed in mobile and embedded devices. The algorithm initially takes into account the characteristics of gesture motion information, integrating the channel attention (CE) mechanism into the pseudo-3D (P3D) module, thereby constructing a P3D-C feature extraction network that can efficiently extract spatio-temporal feature information while reducing the complexity of the algorithmic model. To further enhance the understanding and learning of the global gesture movement's dynamic information, a Multiscale Motion Embedding (MME) mechanism is subsequently designed. The experimental findings reveal that the MME-P3D model achieves recognition accuracies reaching up to 91.12% and 83.06% on the self-constructed conference gesture dataset and the publicly available Chalearn 2013 dataset, respectively. In comparison with the conventional 3D convolutional neural network, the MME-P3D model demonstrates a significant advantage in terms of parameter count and computational requirements, which are reduced by as much as 82% and 83%, respectively. This effectively addresses the limitations of the original algorithms, making them more suitable for deployment on embedded and mobile devices and providing a more effective means for the practical application of hand gesture recognition technology.

    Citation: Hongmei Jin, Ning He, Boyu Liu, Zhanli Li. Research on gesture recognition algorithm based on MME-P3D[J]. Mathematical Biosciences and Engineering, 2024, 21(3): 3594-3617. doi: 10.3934/mbe.2024158

    Related Papers:

    [1] Mahtab Mehrabbeik, Fatemeh Parastesh, Janarthanan Ramadoss, Karthikeyan Rajagopal, Hamidreza Namazi, Sajad Jafari . Synchronization and chimera states in the network of electrochemically coupled memristive Rulkov neuron maps. Mathematical Biosciences and Engineering, 2021, 18(6): 9394-9409. doi: 10.3934/mbe.2021462
    [2] Stefano Cosenza, Paolo Crucitti, Luigi Fortuna, Mattia Frasca, Manuela La Rosa, Cecilia Stagni, Lisa Usai . From Net Topology to Synchronization in HR Neuron Grids. Mathematical Biosciences and Engineering, 2005, 2(1): 53-77. doi: 10.3934/mbe.2005.2.53
    [3] Massimo Fioranelli, O. Eze Aru, Maria Grazia Roccia, Aroonkumar Beesham, Dana Flavin . A model for analyzing evolutions of neurons by using EEG waves. Mathematical Biosciences and Engineering, 2022, 19(12): 12936-12949. doi: 10.3934/mbe.2022604
    [4] Karim El Laithy, Martin Bogdan . Synaptic energy drives the information processing mechanisms in spiking neural networks. Mathematical Biosciences and Engineering, 2014, 11(2): 233-256. doi: 10.3934/mbe.2014.11.233
    [5] Manuela Aguiar, Ana Dias, Miriam Manoel . Gradient and Hamiltonian coupled systems on undirected networks. Mathematical Biosciences and Engineering, 2019, 16(5): 4622-4644. doi: 10.3934/mbe.2019232
    [6] Anna Cattani . FitzHugh-Nagumo equations with generalized diffusive coupling. Mathematical Biosciences and Engineering, 2014, 11(2): 203-215. doi: 10.3934/mbe.2014.11.203
    [7] Guowei Wang, Yan Fu . Spatiotemporal patterns and collective dynamics of bi-layer coupled Izhikevich neural networks with multi-area channels. Mathematical Biosciences and Engineering, 2023, 20(2): 3944-3969. doi: 10.3934/mbe.2023184
    [8] Prasina Alexander, Fatemeh Parastesh, Ibrahim Ismael Hamarash, Anitha Karthikeyan, Sajad Jafari, Shaobo He . Effect of the electromagnetic induction on a modified memristive neural map model. Mathematical Biosciences and Engineering, 2023, 20(10): 17849-17865. doi: 10.3934/mbe.2023793
    [9] Stefano Fasani, Sergio Rinaldi . Local stabilization and network synchronization: The case of stationary regimes. Mathematical Biosciences and Engineering, 2010, 7(3): 623-639. doi: 10.3934/mbe.2010.7.623
    [10] Sridevi Sriram, Hayder Natiq, Karthikeyan Rajagopal, Ondrej Krejcar, Hamidreza Namazi . Dynamics of a two-layer neuronal network with asymmetry in coupling. Mathematical Biosciences and Engineering, 2023, 20(2): 2908-2919. doi: 10.3934/mbe.2023137
  • A Multiscale-Motion Embedding Pseudo-3D (MME-P3D) gesture recognition algorithm has been proposed to tackle the issues of excessive parameters and high computational complexity encountered by existing gesture recognition algorithms deployed in mobile and embedded devices. The algorithm initially takes into account the characteristics of gesture motion information, integrating the channel attention (CE) mechanism into the pseudo-3D (P3D) module, thereby constructing a P3D-C feature extraction network that can efficiently extract spatio-temporal feature information while reducing the complexity of the algorithmic model. To further enhance the understanding and learning of the global gesture movement's dynamic information, a Multiscale Motion Embedding (MME) mechanism is subsequently designed. The experimental findings reveal that the MME-P3D model achieves recognition accuracies reaching up to 91.12% and 83.06% on the self-constructed conference gesture dataset and the publicly available Chalearn 2013 dataset, respectively. In comparison with the conventional 3D convolutional neural network, the MME-P3D model demonstrates a significant advantage in terms of parameter count and computational requirements, which are reduced by as much as 82% and 83%, respectively. This effectively addresses the limitations of the original algorithms, making them more suitable for deployment on embedded and mobile devices and providing a more effective means for the practical application of hand gesture recognition technology.



    The synchronization of complex networks has been one of the most critical issues in recent years [1]. Complex networks, composed of interacting dynamical systems, can be used to represent and analyze real networks [2,3]. The synchronization phenomenon is termed when two or more dynamical systems evolve in common behavior [4]. Numerous natural events can be related to synchronization [1]. Therefore, many studies have focused on the synchronization of coupled oscillators in different fields [5,6]. The synchronizability of a network is strongly associated with the network topology and the coupling between the nodes [4,7]. There are some studies that have considered the synchronization in time-varying networks [8,9]. Belykh et al. [10] showed that adding a few random connections to a network with blinking links enhances synchrony by reducing the required couplings strength for synchronization. Frasca et al. [11] investigated the synchronization of Rossler systems with switching coupling and found that switching between two coupling strengths that cannot separately synchronize the system can lead to synchronization.

    In the nervous system, the communications and interactions among neurons create complex networks [12,13]. Therefore, many of their activities lead to the formation and representation of collective behaviors such as synchronization [14]. Previous studies have revealed the significance of the synchronous behavior of neurons in information processing and cognitive tasks [15]. Subsequently, the study of synchronization in neuronal networks has attracted the attention of many scientists [16,17,18]. Belykh et al. [19] investigated the bursting neurons coupled nonlinearly through chemical synapses and found that synchronization depends on the receiving signals by each neuron. In contrast, synchronization of the electrically coupled neurons relies on the network's topology [20]. Previous studies have considered different network topologies such as small-world, scale-free, modular, hierarchical, etc. [21,22,23]. Furthermore, with the development of the multilayer networks that provide multiple interactions, the synchronization of the neurons was also investigated in this type of topology [8,9]. Some of the studies have considered the factors that can enhance the synchronization in neuronal networks. It has been proved that the time delay [24], the activity-dependent coupling [25], considering the neuron–astrocyte interaction [26], proper adjustment of autapse [27], etc., can help enhance the synchronous behavior among neurons.

    The neuronal functions rely on the communications between them [28,29]. These communications which occur in the cellular regions are called synapses. It is known that the transmissions among neurons can be through electrical or chemical synapses. The electrical synapses occur through channels (gap junctions) when the cytoplasms of two neurons are connected [30]. While in the chemical synapses, the neurotransmitters are released from one cell and received by another [31]. These two types of transmissions exist in different brain regions and are essential for their normal function. Furthermore, Pereda [28] highlighted that the interaction between these two synapses in the healthy brain is also crucial. There is evidence that the electrical synapse transits to chemical synapses in particular brain regions [32,33,34]. In other words, the formation and elimination of electrical and chemical synapses are sequential [35,36]. Szabo et al. [37] represented this serial synaptic evolution in motoneurons from the snail Helisoma. Furthermore, there is a hypothesis that the former electrical synapses are necessary for developing the chemical synapses. Todd et al. [38] represented this fact in the leech neurons.

    Motivated by the above description about the transition of synapses, this paper studies the network of coupled Hindmarsh-Rose neurons coupled with transient synapses. It is assumed that the electrical synapses are initially present, and they are replaced with chemical synapses over time. This sequence of transformations happens with a specific period. The synchronization of the neurons is studied under different conditions by varying the coupling strengths, the period of transitions, and the duration time of the presence of the synapses. The time series of the neurons is also under consideration. Therefore, in the next section, the model, the network, and the coupling are described. The results are presented in the third section, and the conclusions are given in the last section.

    A network of Hindmarsh-Rose neurons with electrical and chemical synapses in a small-world structure is constructed as follows:

    ˙xi=yi+3x2ix3izi+Iext+Nj=1gij(ge(xjxi)+gc(vsxi)1+expexp(λ(xjϵs))),˙yi=15x2iyi,˙zi=r(s(xi+1.6)zi),i=1,,N (1)

    where x,y,z represent the action potential, the fast and slow recovery variables of the neuron. The parameters of the chemical coupling are vs=2, ϵs=0.25,λ=10. gij refers to the network structure such that gij=1 shows the presence of a link between nodes i and j. The coupling strength of the electrical and chemical synapses are ge and gc, respectively. N=100 nodes are connected in a small-world configuration with 20 nearest-neighbor couplings and the probability of 0.1. The parameters of the Hindmarsh-Rose model are set at r=0.006, s=4, Iext=2.8, such that the single neuron exhibits periodic bursting. The coupling between neutrons is through periodic time-varying coupling with electrical and chemical synapses. As discussed before, the synapses are transient and change in the time interval τ. In a part of this time interval (0<t<θτ), the electrical synapses are on, and in the next part (θτ<t<τ), the electrical synapses disappear, and the chemical synapses appear. Therefore, we have:

    0<t<θτ:gc=0,ge0θτ<t<τ:gc0,ge=0 (2)

    The equations of the network are solved numerically by using the fourth-order Runge-Kutta method with a total run time 3000 and time step 0.01. The network is investigated by considering different periods (τ) for the coupling function. The time of presence of the electrical synapses to chemical ones (θ) is also varied. To quantify the synchronization of the neurons, the average synchronization error is computed as follow:

    E=1N1Nj=2(x1xj)2+(y1yj)2+(z1zj)2t, (3)

    At first, we consider that both synapses exist among neurons constantly. The electrical coupling strength is varied, and the synchronization error is computed. Three cases are assumed for the chemical coupling strength as: 1) gc=0.5ge, 2) gc=ge, 3) gc=2ge. The synchronization errors for three cases are illustrated in Figure 1. When the chemical coupling is smaller than the electrical, gc=0.5ge, the neurons become completely synchronous in the region 0.054<ge<0.072. The time series of the synchronous neurons is shown in Figure 2a. As the electrical coupling grows from 0.072, the complete synchronization disappears, and the neurons become burst synchronous. The time series of neurons, in this case, are shown in Figure 2b and represent that although the neurons burst synchronously, there is a small error due to the difference between neurons when they are repolarized. By strengthening the chemical coupling, the synchronization error decays to zero for smaller electrical coupling strength. But the neurons do not oscillate synchronously and reach the resting state. The time series of the neurons in ge=0.2, and for gc=ge, gc=2ge are illustrated in Figure2c and 2d, respectively.

    Figure 1.  The synchronization error of the neurons is coupled with constant electrical and chemical synapses according to the electrical coupling strength. The red, blue, and yellow curves refer respectively to gc=0.5ge, gc=ge, and gc=2ge.
    Figure 2.  The time series of the neuron's action potentials coupled with constant electrical and chemical synapses. (a) The neurons are completely synchronous for ge=0.07 and gc=0.5ge. (b) When gc=0.5ge, the burst synchronization appears for ge>0.072. The resting state is obtained when (c) gc=ge, ge=0.2, and (d) gc=2ge, ge=0.2.

    When the neurons are completely synchronous, all of the variables of the neurons are the same, i.e., x1=x2==xN, y1=y2==yN, z1=z2==zN Therefore, the equation of the synchronous manifold can be derived as

    ˙xi=yi+3x2ix3izi+Iext+gc(vsxi)1+expexp(λ(xiϵs))Nj=1gij,˙yi=15x2iyi,˙zi=r(s(xi+1.6)zi). (4)

    When the chemical coupling strength increases, the coupling term gc(vsxi)1+expexp(λ(xiϵs))Nj=1gij intensifies and overcomes the dynamics of the neurons and leads the neurons to be quiescent.

    Next, we investigate the case of transient synapses. A specific period (τ) is considered, and in a part of the period, the electrical synapses are on (gc=0), and in the next part, the synapses are replaced with the chemical ones (ge=0). Firstly, it is assumed that the duration of both synapses is the same, i.e., θ=0.5, and the effect of the transition period is studied. As mentioned in the previous section, with the parameters selected for the Hindmarsh-Rose model, the single neuron bursts periodically, and the period of bursts is T=132. Thus, the synaptic transition period is considered to be the coefficients of the bursts periods as τ=33,66,132,264,396, and 528. The synchronization errors for different periods are presented in Figure 3. Similar to the previous case, as the chemical coupling strength increases, the error reaches its minimum in more miniature electrical couplings. This can be inferred from comparing the error curves in parts a to c. Furthermore, in all cases, the errors do not reach zero, and there is a minimum error which shows that the neurons are burst synchronized. This minimum error becomes larger by increasing the period of synaptic transitions. Moreover, it leads to a change in the waveform of the oscillations and the frequency of the bursts.

    Figure 3.  The synchronization error of the neurons coupled with transient electrical and chemical synapses with θ=0.5 according to the electrical coupling strength. (a) gc=0.5ge, (b) gc=ge, (c) gc=2ge. Different colors represent the errors for different synaptic transition periods. With the increase of the period, the final synchronization error increases.

    The time series of all neurons for different periods are shown in Figure 4. Although the parameters of the neurons were set in such a way to show square-wave bursting, adding the time-varying coupling term alters the neurons' firing pattern. It can be observed that for lower period values, i.e., for τ=33 (Figure 4a) and τ=66 (Figure 4b), the pattern consists of a single spike followed by a relaxation oscillation with short duration, which can be called pseudo-plateau burst. By increasing the period, the pattern changes to relaxation oscillation (for τ=132,264 in Figure 4c, 4d). Raising the period to τ=396 leads to the pattern to be constructed by multiple rapid spikes and a depolarized silence that is known as circle/fold cycle bursters or depolarization block bursting (Figure 4e) [39]. For τ=528, it seems that each burst is composed of both plateau burst type and pseudo-plateau burst type (Figure4f). As mentioned, the neurons are not fully synchronized, and there is a small error that refers to the depolarized silence duration. Figure 4g shows the enlargement of one burst of neurons relating to part e. The difference between neurons in the depolarized silence duration is obvious in this figure.

    Figure 4.  The time series of the neuron's action potentials coupled with transient electrical and chemical synapses with θ=0.5 for the different synaptic transition periods. (a)τ=33, (b) τ=66, (c) τ=132, (d) τ=264, (e) τ=396, (f) τ=528, (g) enlargement of a burst shown by red box in part (e). By increasing τ, the firing pattern changes, and the frequency of the bursts decreases.

    With the increase in the period of synaptic transitions, the bursts become longer, and the frequency of the bursts decreases. Comparing the period of bursts with the transition period shows that for longer periods, i.e., τ=264,396,528, the period of the bursts is equal to the transition period (τ). For shorter periods as τ=66,132, the period of the bursts is twice the transition period (2τ). For shorter periods, there is no relation between these periods. Furthermore, the spikes refer to the presence of the electrical synapses, while the silence mode refers to the time of the chemical synapses. Figure 5 represents this issue for τ=66,396. In these figures, the parts of the time series related to the presence of the electrical and chemical synapses are shown by red and blue colors, respectively.

    Figure 5.  The time series of the neuron's action potentials coupled with transient electrical and chemical synapses with θ=0.5 and (a)τ=66, (b) τ=396. The parts of the time series related to the presence of the electrical and chemical synapses are shown by red and blue colors, respectively.

    Finally, the influence of the time of existence of the electrical and chemical synapses is studied. To this aim, the parameter θ is varied. For example, considering θ=0.1 means that in 0.1τ, the electrical synapses are on, and in the rest of 0.9τ, the chemical synapses are present. Figure 6 represents the synchronization error for τ=132 and θ=0.1,0.3,0.5,0.7, and 0.9 for three cases of chemical coupling strength. It can be observed that similar to the previous cases, the increase of the chemical coupling leads to decay of error in lower electrical strengths. However, the critical electrical coupling for synchronization doesn't depend on the value of chemical strength for θ=0.9, in which the chemical synapses exist in a short interval (0.1τ). Furthermore, as the time of electrical synapses to chemical synapses increases, the final synchronization error decreases. For θ<0.9, by increasing ge, there is a minimum error, and the neurons become burst synchronous. While for θ=0.9, the zero error is obtained, and complete synchronization can be attained. The time series of the neurons for different θ are shown in Figure 7. It can be observed that the time of the presence of synapses also impacts the waveform of the action potential.

    Figure 6.  The synchronization error of the neurons coupled with transient electrical and chemical synapses for τ=132 according to the time duration of the presence of synapses (θ). (a) gc=0.5ge, (b) gc=ge, (c) gc=2ge. Different colors represent the errors for different time ratios of synapses. As θ increases, the final synchronization error decreases. The complete synchronization is obtained for θ=0.9.
    Figure 7.  The time series of the neuron's action potentials coupled with transient electrical and chemical synapses for τ=132, and different timings for synapses. (a) θ=0.1, (b)θ=0.3, (c) θ=0.5, (d) θ=0.7, (e) θ=0.9. Varying θ effects on the shape of time series.

    Figure 8 presents a complete view of the synchrony error of neurons in the parameter planes by varying the transition period and the timing of synapses for gc=ge. The effects of changing the transitions period (τ) are demonstrated in part (a) for θ=0.5. It is observed that increasing τ leads to burst synchronization occurring in a slightly lower coupling strength ge. However, for higher τ, the final synchronization error by increasing ge is more significant. Part (b) of this figure illustrates the impact of varying the presence time of electrical and chemical synapses (θ) for a fixed transition period τ=132. It is attained that in very low values of θ (θ<0.1), i.e. when the present time of electrical synapses is short, the synchrony error does not decay to zero by increasing ge. For 0.1<θ<0.5, the error becomes zero for intermediate values of coupling strength, and there exists a small error when the coupling becomes stronger. As the present time of electrical synapses increases (θ>0.5), this small error does not appear for strong couplings.

    Figure 8.  Synchronization error of the neurons with transient synapses. (a) By varying ge and τ for θ=0.5: Synchronization is disturbed by increasing the transition period. (b) By varying ge and θ for τ=132: Synchronization is enhanced by increasing the presence time of electrical synapses.

    This paper investigated the effects of the transient electrical synapses on the synchronization of the coupled neurons. Previous studies on the brain have revealed that the electrical synapses transit to the chemical synapses in special brain regions. Motivated by this, it was assumed that the electrical synapses exist among neurons for a defined time interval and then change to chemical synapses in the next time interval, and this sequence repeats periodically. The synchronization error of the neurons was computed by considering different periods for synaptic transitions and different time durations for synapses.

    At first, the network with constant electrical and chemical synapses was investigated. It was observed that when the chemical coupling was weak, the neurons were completely synchronous in a specific range of electrical coupling strength, further which the burst synchronization emerged. As the chemical coupling became more robust, the oscillations were suppressed, and the neurons stayed in a resting state. When the synapses were transient, the neurons were oscillating in all cases, and the steady resting state was not observed. The results showed that complete synchronization was only obtained when the time duration of electrical synapses was high. In other cases, the burst synchronization was observed. Increasing the period of the synaptic transitions led to the increase of the absolute synchronization error. In all cases, increasing the chemical coupling caused the decay of the synchronization error in lower electrical strengths. Moreover, both the period and the time duration of the electrical and chemical synapses influenced the shape of the time series of the action potentials.

    This work is supported by the Natural Science Basic Research Program of Shaanxi (2021JM-533, 2021JQ-880, 2020JM-646), the Innovation Capability Support Program of Shaanxi (2018GHJD-21), the Science and Technology Program of Xi'an (2019218414GXRC020CG021-GXYD20.3), the Support Plan for Sanqin Scholars Innovation Team in Shaanxi Province of China, the Scientific Research Program Funded by Shaanxi Provincial Education Department (21JK0960), the Youth Innovation Team of Shaanxi Universities, the Scientific Research Foundation of Xijing University (XJ21B01), and the Scientific Research Foundation of Xijing University (XJ200202). This work is partially funded by Centre for Nonlinear Systems, Chennai Institute of Technology, India vide funding number CIT/CNS/2021/RD/064.

    The authors declare that they have no conflict of interest.



    [1] Y. Zhang, J. Wang, X. Wang, H. Jing, Z. Sun, Y. Cai, Static hand gesture recognition method based on the vision transformer, Multimedia Tools Appl., 82 (2023), 1–20. https://doi.org/10.1007/s11042-023-14732-3 doi: 10.1007/s11042-023-14732-3
    [2] T. Zhang, Application of AI-based real-time gesture recognition and embedded system in the design of English major teaching, Wireless Netw., 2021 (2021), 1–13. https://doi.org/10.1007/s11276-021-02693-0 doi: 10.1007/s11276-021-02693-0
    [3] Y. Xue, Y. Yu, K. Yin, P. Li, S. Xie, Z. Ju, Human in-hand motion recognition based on multi-modal perception information fusion, IEEE Sens. J., 22 (2022), 6793–6805. https://doi.org/10.1109/JSEN.2022.3148992 doi: 10.1109/JSEN.2022.3148992
    [4] M. S. Amin, S. T. H. Rizvi, M. M. Hossain, A comparative review on applications of different sensors for sign language recognition, J. Imaging, 8 (2022), 1–48. https://doi.org/10.3390/jimaging8040098 doi: 10.3390/jimaging8040098
    [5] Y. Zhang, Y. Huang, X. Sun, Y. Zhao, X. Guo, P. Liu, et al., Static and dynamic human arm/hand gesture capturing and recognition via multiinformation fusion of flexible strain sensors, IEEE Sens. J., 20 (2020), 6450–6459. https://doi.org/10.1109/JSEN.2020.2965580 doi: 10.1109/JSEN.2020.2965580
    [6] N. H. Dardas, N. D. Georganas, Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques, IEEE Trans. Instrum. Meas., 60 (2011), 3592–3607. https://doi.org/10.1109/TIM.2011.2161140 doi: 10.1109/TIM.2011.2161140
    [7] B. Qiang, Y. Zhai, M. Zhou, X. Yang, B. Peng, Y. Wang, et al., SqueezeNet and fusion network-based accurate fast fully convolutional network for hand detection and gesture recognition, IEEE Access, 9 (2021), 77661–77674. https://doi.org/10.1109/ACCESS.2021.3079337 doi: 10.1109/ACCESS.2021.3079337
    [8] P. Barros, S. Magg, C. Weber, S. Wermter, A multichannel convolutional neural network for hand posture recognition, in Artificial Neural Networks and Machine Learning–ICANN 2014: 24th International Conference on Artificial Neural Networks, Hamburg, Germany, September 15–19, 2014. Proceedings 24, (2014), 403–410. https://doi.org/10.1007/978-3-319-11179-7_51
    [9] S. Gnanapriya, K. Rahimunnisa, A hybrid deep learning model for real time hand gestures recognition, Intell. Autom. Soft Comput., 36 (2023), 763–767. https://doi.org/10.32604/iasc.2023.032832 doi: 10.32604/iasc.2023.032832
    [10] Q. Miao, Y. Li, W. Ouyang, Z. Ma, X. Xu, W. Shi, et al., Multimodal gesture recognition based on the resc3d network, in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), (2017), 3047–3055. https://doi.org/10.1109/ICCVW.2017.360
    [11] D. Tran, L. Bourdev, R. Fergus, L. Torresani, M. Paluri, Learning spatiotemporal features with 3D convolutional networks, in 2015 IEEE International Conference on Computer Vision (ICCV), (2015), 4489–4497. https://doi.org/10.1109/TIP.2021.3092828
    [12] J. Wan, S. Escalera, G. Anbarjafari, H. Jair Escalante, X. Baró, I. Guyon, et al., Results and analysis of chalearn lap multi-modal isolated and continuous gesture recognition, and real versus fake expressed emotions challenges, in Proceedings of the IEEE International Conference on Computer Vision Workshops, (2017), 3189–3197. https://doi.org/10.1109/ICCVW.2017.377
    [13] Z. Z. Wang, Automatic and robust hand gesture recognition by SDD features based model matching, Appl. Intell., 52 (2022), 11288–11299. https://doi.org/10.1007/s10489-021-02933-y doi: 10.1007/s10489-021-02933-y
    [14] Q. Gao, Y. Chen, Z. Ju, Y. Liang, Dynamic hand gesture recognition based on 3D hand pose estimation for human–robot interaction, IEEE Sens. J., 18 (2021), 17421–17430. https://doi.org/10.1109/JSEN.2021.3059685 doi: 10.1109/JSEN.2021.3059685
    [15] Y. Li, D. Zhang, J. Chen, J. Wan, D. Zhang, Y. Hu, et al., Towards domain-independent and real-time gesture recognition using mmwave signal, IEEE Trans. Mob. Comput., 22 (2022), 7355–7369. https://doi.org/10.1109/TMC.2022.3207570 doi: 10.1109/TMC.2022.3207570
    [16] M. Wang, X. Li, S. Chen, X. Zhang, L. Ma, Y. Zhang, Learning representations by contrastive spatio-temporal clustering for skeleton-based action recognition, IEEE Trans. Multimedia, 2023 (2023), 1–14. https://doi.org/10.1109/TMM.2023.3307933 doi: 10.1109/TMM.2023.3307933
    [17] C. Pang, X. Gao, Z. Chen, L. Lyu, Self-adaptive graph with nonlocal attention network for skeleton-based action recognition, IEEE Trans. Neural Networks Learn. Syst., 2023 (2023), 1–13. https://doi.org/10.1109/TNNLS.2023.3298950 doi: 10.1109/TNNLS.2023.3298950
    [18] P. Geng, X. Lu, C. Hu, H. Liu, L. Lyu, Focusing fine-grained action by self-attention-enhanced graph neural networks with contrastive learning, IEEE Trans. Circuits Syst. Video Technol., 33 (2023), 4754–4768. https://doi.org/10.1109/TCSVT.2023.3248782 doi: 10.1109/TCSVT.2023.3248782
    [19] W. Song, T. Chu, S. Li, N. Li, A. Hao, H. Qin, Joints-centered spatial-temporal features fused skeleton convolution network for action recognition, IEEE Trans. Multimedia, 2023 (2023), 1–15. https://doi.org/10.1109/TMM.2023.3324835 doi: 10.1109/TMM.2023.3324835
    [20] C. Pang, X. Lu, L. Lyu, Skeleton-based action recognition through contrasting two-stream spatial-temporal networks, IEEE Trans. Multimedia, 25 (2023), 8699–8711. https://doi.org/10.1109/TMM.2023.3239751 doi: 10.1109/TMM.2023.3239751
    [21] C. Xu, X. Wu, M. Wang, F. Qiu, Y. Liu, J. Ren, Improving dynamic gesture recognition in untrimmed videos by an online lightweight framework and a new gesture dataset ZJUGesture, Neurocomputing, 523 (2023), 58–68. https://doi.org/10.1016/j.neucom.2022.12.022 doi: 10.1016/j.neucom.2022.12.022
    [22] Z. Qiu, T. Yao, T. Mei, Learning spatio-temporal representation with pseudo-3D residual networks, in Proceedings of the IEEE International Conference on Computer Vision, (2017), 5533–5541. https://doi.org/10.1109/ICCV.2017.590
    [23] D. Tran, H. Wang, L. Torresani, J. Ray, Y. LeCun, M. Paluri, A closer look at spatiotemporal convolutions for action recognition, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 6450–6459. https://doi.org/10.1109/CVPR.2018.00675
    [24] S. Xie, C. Sun, J. Huang, Z. Tu, K. Murphy, Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification, in Proceedings of the European Conference on Computer Vision (ECCV), (2018), 318–335. https://doi.org/10.1007/978-3-030-01267-0_19
    [25] Z. Huang, X. Shi, C. Zhang, Q. Wang, K. C. Cheung, Flowformer: A transformer architecture for optical flow, in European Conference on Computer Vision, (2022), 668–685. https://doi.org/10.1007/978-3-031-19790-1_40
    [26] J. Wang, X. Li, J. Li, Q. Sun, H. Wang, NGCU: A new RNN model for time-series data prediction, Big Data Res., 27 (2022), 100296. https://doi.org/10.1016/j.bdr.2021.100296 doi: 10.1016/j.bdr.2021.100296
    [27] T. Huynh-The, C. H. Hua, N. A. Tu, D. S. Kim, Learning 3D spatiotemporal gait feature by convolutional network for person identification, Neurocomputing, 397 (2020), 192–202. https://doi.org/10.1016/j.neucom.2020.02.048 doi: 10.1016/j.neucom.2020.02.048
    [28] B. Zhou, C. Han, T. Guo, Convergence of stochastic gradient descent in deep neural network, Acta Math. Appl. Sin., 37 (2021), 126–136. https://doi.org/10.1007/s10255-021-0991-2 doi: 10.1007/s10255-021-0991-2
    [29] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, et al., Mobilenets: Efficient convolutional neural networks for mobile vision applications, preprint, arXiv: 1704.04861. https://doi.org/10.48550/arXiv.1704.04861
    [30] J. Carreira, A. Zisserman, Quo vadis, action recognition? a new model and the kinetics dataset, preprint, arXiv: 1705.07750. https://doi.org/10.48550/arXiv.1705.07750
    [31] H. Kataoka, T. Wakamiya, K. Hara, Y. Satoh, Would mega-scale datasets further enhance spatiotemporal 3D CNNs, preprint, arXiv: 2004.04968. https://doi.org/10.48550/arXiv.2004.04968
  • This article has been cited by:

    1. Hongliang Miao, Amit Gupta, Coordinated Development of China’s Regional Economy and Ethnic Diversity under the Background of Big Data and the Internet of Things, 2022, 2022, 1875-905X, 1, 10.1155/2022/6424505
    2. Yule Wang, Arwa Abdulkreem AL‐Huqail, Shadi Salimimoghadam, Khidhair Jasim Mohammed, Amin Jan, H. Elhosiny Ali, Mohamed Amine Khadimallah, Hamid Assilzadeh, The metaheuristic optimization of the mechanical properties of sustainable energies using artificial neural networks and genetic algorithm: A case study by eggshell fine waste, 2022, 46, 0363-907X, 21338, 10.1002/er.8255
    3. Khaled Benkouider, Sundarapandian Vaidyanathan, Aceng Sambas, Esteban Tlelo-Cuautle, Ahmed A. Abd El-Latif, Bassem Abd-El-Atty, Ciro Fabian Bermudez-Marquez, Ibrahim Mohammed Sulaiman, Aliyu Muhammed Awwal, Poom Kumam, A New 5-D Multistable Hyperchaotic System With Three Positive Lyapunov Exponents: Bifurcation Analysis, Circuit Design, FPGA Realization and Image Encryption, 2022, 10, 2169-3536, 90111, 10.1109/ACCESS.2022.3197790
    4. Linsen Li, Wen-Tsao Pan, Node Location Method of Ecological Environment Monitoring Network Based on Zigbee, 2022, 2022, 1607-887X, 1, 10.1155/2022/5854569
    5. Hong Huo, Huanning Xu, Arpit Bhardwaj, Construction of Emergency Procurement System and System Improvement Based on Convolutional Neural Network, 2022, 2022, 1687-5273, 1, 10.1155/2022/6139706
    6. Qian Wang, Wen-Tsao Pan, Population Economic Transfer Function Model Based on Genetic Tabu Search Algorithm, 2022, 2022, 1607-887X, 1, 10.1155/2022/1356909
    7. Liping Chen, Amit Gupta, Coordinated Development of Smart City and Regional Industrial Economy under the Background of Internet of Things, 2022, 2022, 1875-905X, 1, 10.1155/2022/6986090
    8. K. Immanuvel Arokia James, R. Prabakaran, A. Karthikeyan, R. R. Prianka, Co-operative beam forming selection with energy balanced operation for wireless sensor network, 2022, 28, 1022-0038, 3653, 10.1007/s11276-022-03067-w
    9. Nimet Korkmaz, İbrahim Ethem Saçu, An alternative perspective on determining the optimum fractional orders of the synaptic coupling functions for the simultaneous neural patterns, 2022, 110, 0924-090X, 3791, 10.1007/s11071-022-07782-z
    10. Fang Li, Tao Li, Wen-Tsao Pan, Tourism Consumer Demand Forecasting under the Background of Big Data, 2022, 2022, 1563-5147, 1, 10.1155/2022/4335718
    11. Kimia Latifi, Ahoo Ebrahimi, Mehdi Ranjbaran, Arman Mirzaei, Zahra Fakhri, Efficient customer relationship management systems for online retailing: The investigation of the influential factors, 2022, 1833-3672, 1, 10.1017/jmo.2022.65
    12. Tingting Cai, Dongmin Yu, Huanan Liu, Fengkai Gao, Computational Analysis of Variational Inequalities Using Mean Extra-Gradient Approach, 2022, 10, 2227-7390, 2318, 10.3390/math10132318
    13. Lan Zheng, Ning Cao, Predictive Control of the Mobile Robot under the Deep Long-Short Term Memory Neural Network Model, 2022, 2022, 1687-5273, 1, 10.1155/2022/1835798
    14. Caixia Wu, Zhenrong Zhao, Yuanyuan Liu, Bayan Omar Mohammed, Quantum-dot cellular automata-based design for three-level nanoscale full-subtractor, 2022, 05779073, 10.1016/j.cjph.2022.10.014
    15. Shanshan Yu, Hao Wang, Yajun Wang, Pradeep Tomar, Optimization Design of Street Public Space Layout on Account of Internet of Things and Deep Learning, 2022, 2022, 1687-5273, 1, 10.1155/2022/7274525
    16. Eman A. Atta, Ahmed F. Ali, Ahmed A. Elshamy, Kathiravan Srinivasan, A modified weighted chimp optimization algorithm for training feed-forward neural network, 2023, 18, 1932-6203, e0282514, 10.1371/journal.pone.0282514
    17. Seyed Sajad Ahmadpour, Nima Jafari Navimipour, Mohammad Mosleh, Senay Yalcin, Nano-design of ultra-efficient reversible block based on quantum-dot cellular automata, 2023, 24, 2095-9184, 447, 10.1631/FITEE.2200095
    18. Prasina Alexander, Fatemeh Parastesh, Ibrahim Ismael Hamarash, Anitha Karthikeyan, Sajad Jafari, Shaobo He, Effect of the electromagnetic induction on a modified memristive neural map model, 2023, 20, 1551-0018, 17849, 10.3934/mbe.2023793
    19. Arash Heidari, Nima Jafari Navimipour, Hasan Dag, Mehmet Unal, Deepfake detection using deep learning methods: A systematic and comprehensive review, 2024, 14, 1942-4787, 10.1002/widm.1520
    20. Gabrielle S. Prince, Molly Reynolds, Verdion Martina, HaoSheng Sun, Gene-environmental regulation of the postnatal post-mitotic neuronal maturation, 2024, 40, 01689525, 480, 10.1016/j.tig.2024.03.006
    21. Xiaodi Li, Ying Xu, Energy level transition and mode transition in a neuron, 2024, 112, 0924-090X, 2253, 10.1007/s11071-023-09147-6
    22. Xinqing Duan, Lei Li, Zehui Peng, Mingqiang Wang, Yanxin Liu, Dar‐Jen Hsieh, Kuan‐Chang Chang, Ultralow Power, Cleft Size‐Adjustable and pH‐Sensitive Hyaluronic Acid (HA) Biodevices for Acid‐Sensing Ion Channels Emulation, 2024, 1613-6810, 10.1002/smll.202405207
    23. Jindong Liu, Zhen Wang, Huaigu Tian, Fei Xie, Zeljko Stevic, Complex Dynamic Analysis, Circuit Design and Simplified Predefined Time Synchronization for a Jerk Absolute Memristor Chaotic System, 2023, 2023, 1099-0526, 1, 10.1155/2023/5912191
    24. Xinlin Song, Feifei Yang, A light-temperature neuron and its adaptive regulation, 2024, 99, 0031-8949, 125247, 10.1088/1402-4896/ad8fe4
    25. Mahtab Mehrabbeik, Fatemeh Parastesh, Karthikeyan Rajagopal, Sajad Jafari, Matjaz Perc, Riccardo Meucci, Minimal universal laser network model: Synchronization, extreme events, and multistability, 2024, 13, 2192-8029, 10.1515/nleng-2024-0044
    26. Zhenhua Yu, Kailong Zhu, Ya Wang, Feifei Yang, Dynamics of a neuron with a hybrid memristive ion channel, 2025, 194, 09600779, 116233, 10.1016/j.chaos.2025.116233
    27. Xiaowei Han, Hagiwara Akifumi, Pin Lv, Jiahuan Liu, Xiaowei Huang, Renyuan Liu, Xiaojing Long, Yang Liu, Jiangong Zhang, Guolin Ma, Bing Zhang, Low‐Intensity Focused Ultrasound Ameliorates Cisplatin‐Induced Cognitive Impairment by Attenuating Hippocampal Neuroinflammation and Enhancing Synaptic Plasticity in Rats, 2025, 2834-2860, 10.1002/ird3.70003
  • Reader Comments
  • © 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(1283) PDF downloads(81) Cited by(0)

Figures and Tables

Figures(16)  /  Tables(4)

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog