Improved wolf swarm optimization with deep-learning-based movement analysis and self-regulated human activity recognition

1.   Introduction

The goal of human activity recognition (HAR) is to identify activities from a collection of data on subject behavior and environmental factors. Computer vision and human-computer interaction experts have been working on HAR for the last three decades, proposing several methods and techniques for improving the process. The earliest HAR-related work was done in the late 1990s. A lot of recent research has been devoted to testing out methods for distinguishing activities of daily life (ADLs) from inertial signals. The widespread use of mobile devices with inertial sensors and the falling cost of hardware are the key causes of this. Smart homes, surveillance, healthcare and other application contexts can all benefit from the use of smartphones that can acquire and process signals. The advancement of new systems with assistive and medical techniques to create an appropriate environment (provide living conditions via the environment) or provide long-term care shows that research workers are exploring the living standard of elder people and their individuality ^[1]. HAR is an innovative technology that could identify human activities via computer systems and sensors. A HAR system is complicated and could monitor individual situations and provide effective devices in case of an emergency ^[2]. Activity represents a behavior that comprises a set of activities done by interacting with one another or one individual. Providing appropriate and accurate data regarding the activity is the major computation task in the HAR technique ^[3]. With the increased development of neural networks, computing and machine learning algorithms, HAR based on wearable sensors becomes widespread in different areas involving medical services, smart homes, healthcare for old age people ^[4], mechanization in industry, enlightening human interface with computers, security systems, robot monitoring system, monitoring athlete training and rehabilitation systems. It is categorized into 3 classes in data acquisition: wearable sensors, external sensors (non-wearable) and an incorporation of the 2 abovementioned classes ^[5].

Due to the advancement in machine learning and context-aware technologies, researchers have employed distinct techniques for HAR using data gathered from smartphones ^[6]. Smartphones have become more popular for HAR because of three reasons. The first is the ubiquitous nature of this smaller device that is utilized by almost everybody. The next is the efficiency and reliability of the data procurement, and finally, less restriction is considered with respect to privacy concerns. Thus, many kinds of research were introduced through distinct artificial intelligence (AI) technologies ^[7]. Lately, deep learning (DL) has revolutionized conventional machine learning (ML) and brought about enhanced performance in various domains involving object detection, image recognition, natural language processing and speech recognition.

DL has enhanced the robustness and performance of HAR, which speeds up its application and adoption to a wider variety of wearable sensor-based applications ^[8]. There exist two major reasons why DL is robust for a variety of applications. First, the DL method is skilled at directly learning strong features from raw information for certain uses, while the feature needs to be manually engineered or extracted in a conventional ML technique that generally needs a large amount of human effort and proficient domain knowledge ^[9]. Deep neural networks (DNN) could effectively learn representative features from raw signals with smaller domain knowledge. Any neural network architecture, mathematically speaking, seeks to identify any mathematical function y = f(x) that can map attributes(x) to output (y) ^[10]. DL has witnessed considerable development in HAR-based applications due to this expressive power.

This paper presents an improved wolf swarm optimization with deep learning based movement analysis and self-regulated human activity recognition (IWSODL-MAHAR) method. The IWSODL-MAHAR approach aimed to recognize various kinds of human activities. The IWSO algorithm is applied as a dimensionality reduction technique. In addition, the IWSODL-MAHAR technique uses a hybrid DL model for activity recognition. To further improve the recognition performance, a Nadam optimizer is applied as a hyperparameter tuning technique. The experimental evaluation of the IWSODL-MAHAR technique is performed with benchmark activity recognition datasets.

Hassan et al. ^[11] introduced a mobile device inertial sensor-related methodology for HAR. In this study, efficient features will be first mined from the raw datasets. The features involve autoregressive coefficients, mean, median and so on. To make them more robust, the features are processed by linear discriminant analysis (LDA) and kernel principal component analysis (KPCA). At last, the features will be trained with deep belief networks (DBN) for effective activity recognition. In ^[12], a method for HAR utilizing DL was modeled related to stepped frequency continuous wave (SFCW) radar. To be specific, SFCW radar was mainly employed for generating 2 types of characteristic representation domains, as they are range maps in the range domain and multiple frequencies of spectrograms in the time–frequency domain. After that, a particular DL network, which includes a sparse auto encoders (AE) and multiple parallel deep convolutional neural network (DCNN), was modeled for extracting and fusing such features linked with human actions from range maps and multifrequency spectrograms.

In ^[13], the authors focused on the DL-enhanced HAR in Internet of healthcare things (IoHT) atmospheres. A semi-supervised DL structure was devised and framed for precise HAR which effectively employs and examines the weakly labeled sensor datasets for training the classifier learning method. An intellectual autolabeling technique related to deep Q-network (DQN) can be formulated with new modeled distance-oriented reward rules, to better solve the issue of inadequately labeled samples that enhances the learning efficiency in Internet of things (IoT) platforms. Gumaei et al. ^[14] introduced a potential multi-sensor-oriented structure for HAR utilizing a hybrid DL method, which integrates the gated recurrent unit (GRU) and a simple recurrent unit (SRU) of NN. The authors employ deep SRUs for processing a series of multimodal input datasets by utilizing the ability of their internal memory states. Chen et al. ^[15] introduced an innovative DL related method, that is, attention oriented BLSTM (ABLSTM), for passive HAR utilizing WiFi channel state information (CSI) signals. From raw sequential CSI readings, representative features in two directions were learned using the BLSTM. The authors use an attention system for assigning different weights for every learned feature.

Thakur et al. ^[16] proposed a DL-related method for activity recognition with smartphone sensor dataset, that is, gyroscope dataset and accelerometer. Long short-term memory networks (LSTM), convolutional neural network (CNN), and auto encoders (AE) possess complementary modeling capabilities, as LSTMs are adept at temporal modeling, CNNs are suitable for automated feature extraction and AEs were employed for dimensionality reduction. Wan et al. ^[17] presented a smartphone inertial accelerometer-oriented structure for HAR. Whenever the participants do day-to-day activities, the smartphone will collect the sensory data series, derive the high-efficiency features from the original datasets and acquire physical behavior data of users by utilizing multiple three-axis accelerometers. Moreover, a real-time human activity classifier technique relies upon a CNN that was modeled, which leverages CNNs for local feature extracting purposes.

2.   Materials and methods

In this paper, an automated IWSODL-MAHAR algorithm has been modeled for movement and action detection. The major intention of the IWSODL-MAHAR technique is to recognize various kinds of human activities. At the primary level, the activity data is preprocessed to transform it into a meaningful format. Next, the dimensionality reduction process is carried out using the IWSO algorithm. Finally, the hybrid DL model is applied to the activity recognition process.

2.1.   Data preprocessing

In this study, the activity data is primarily preprocessed to transform it into a meaningful format. The standard scalar approach is used to remove the mean and scale the data into unit variance. The major concept behind the standard scalar approach is that it converts the data so that the distribution holds an average value of 0 and a standard deviation of 1. For multivariate data, the preprocessing occurs feature-wise (in other words, independently for every column of data). For a provided data distribution, every individual value in the dataset has the mean value deducted and is divided by the standard deviation of the entire dataset.

2.2.   Dimensionality reduction process

To reduce the high dimensionality of data, the IWSO algorithm is utilized in this work. The IWSO technique is an optimum search method developed for simulating prey allocation, wolves' division of labor and cooperative hunting ^[18,19]. It has the features of fast convergence and global search. In comparison to WSO, the improvement of IWSO has been demonstrated in the following. In roaming behavior, we present the wolf detection updating rule and design a "coarse to fine" roaming model. The fundamental concept is shown in the following: In the $h$ direction around wolf detective $i$, if the function value in the $h$ direction is small when compared to the main function values of wolf detective $i$, then it is likely that there will be a maximum value near the wolf detective $i$, and the tour was reduced.

In Eq (1), $p$ refers to the direction, $d$ denotes the dimension, $ste{p}_{a}$ represent the walking step length, $k$ shows the iteration count, and $\eta \in (-\mathrm{0.1, 0.1})$, $\varTheta \in (0, p/{l}_{i}),$ $h\in \left[\mathrm{4, 7}\right]$ are integers. In comparison to the conventional WSO, the IWSO has the following features: IWSO technique has strong global search capability and escapes from local optima; the proposal of siege radius improves the local development capability and quantifies the algorithm to escape from the local optima; in the solution of 2D functions, the calculation accurateness and convergence speed of IWSO were high; in the solution of calculation speed, the multidimensional complex function, the convergence speed and calculation accuracy of IWSO were more efficient. Figure 1 illustrates the steps involved in IWSO technique.

The computation step was shown below:

Step 1: Initialization.

As per the reverse bidirectional chaotic update approach, the maximal count of iterations ${k}_{\mathrm{m}\mathrm{a}\mathrm{x}},$ number $N$ of artificial wolves, distance determination factors $\omega ,$ maximal number of walks Tmax, wolf group spatial position X, update scale factor $\beta$ and step factor $S$ are initialized.

Step 2: Wandering behavior.

The artificial wolf with large main function value was chosen as the head wolf, and the remaining artificial wolves were considered detective wolves and walk based on Eq. (1) unless the ${Y}_{i}$ objective function value of $i$ - $th$ wolf detective was larger than ${Y}_{Lead}$ main function value of leading wolves

Step 3: Summoning behavior.

The Alpha Wolf was named, and Detective Wolves quickly rushed to Alpha Wolf as follows:

When the ${Y}_{i}$ function values of $ith$ wolf are greater than the ${Y}_{lead}$ function value of the head wolf, return to the end of Step 2; when ${Y}_{i}$ function values of $i$ - $th$ wolve is lesser than function value ${Y}_{lead}$ of head wolve, the $i$ - $th$ wolf detective continue to attack unless it enters siege region, viz., distance ${L}_{i}$ among $i$ - $th$ wolf detective and head, wolf is lesser than or equivalent to ${L}_{lead}$, and return to Step 4.

Step 4: Siege behaviour.

The location of the head wolves is considered the prey location, and wolves contributing in the siege would besiege the prey.

Step 5: Wolves update.

The objective function value of the optimum wolf produced in these iterations is compared to the main function values of head wolves in the preceding iteration. Define the number $R$ of artificial wolves with a small value of the main function to be removed based on the updating scale factor $\beta$. When $t$ is lesser than ${t}_{\mathrm{m}\mathrm{a}\mathrm{x}},$ the wolf group would be upgraded based on the subsequent equation and integrated with reverse learning; then, the wolf group would be upgraded based on the reverse double chaos strategy:

where $\in (-\mathrm{0.1, 0.1})$.

Step 6: Judgment ended.

Decide whether the head wolf's objective function has fulfilled the requirement of computational performance, or the process obtains the maximal amount of iterations ${k}_{\mathrm{m}\mathrm{a}\mathrm{x}}$. Then, the output is the location of the head wolf, that is, the objective function value, or else, return to Step 2.

The fitness function (FF) employed in the presented technique was modeled to maintain a balance between the classifier accuracy (max) gained through these selected features and the number of selected features in all solutions (min). Eq (4) signifies the FF to assess solutions.

where ${\gamma }_{R}\left(D\right)$ signifies the classifier error rate of a given classifier was exploited here. $\alpha$ and $\beta$ were 2 parameters equivalent to the significance of classification quality and subset length $, \ \ \left|C\right|$ denotes total number of features in the dataset $, \left|R\right|$ indicates cardinality of the selected subset $\beta = 1-\alpha .$

The IWSO algorithm involves three types of wolves: alpha, beta and omega. The alpha wolf is the best solution found so far, the beta wolf is the second-best solution, and the omega wolf is the third-best solution. The algorithm starts with a random population of wolves and iteratively improves the solutions by following the hunting behavior of wolves.

Algorithm 1: IWSO algorithm Pseudo-code

Initialize population of wolves

Calculate fitness of each wolf

Set global best wolf as the wolf with the highest fitness

repeat until termination criteria are met:

for each wolf:

Determine the three neighboring wolves: alpha, beta and delta

Generate a new position for the wolf using the formula:

new_position = current_position + (alpha_position - (A * D)) * r1

+ (beta_position - (B * D)) * r2

+ (delta_position - (C * D)) * r3

Compute the new position's suitability.

If the new position is better than the current position, update the wolf's position and

fitness

If the new position is the best among all wolves, update the global best position

Update the values of A, B, C and D for the next iteration

Return the best wolf and its fitness

2.3.   Activity recognition process

For activity recognition, the IWSODL-MAHAR technique makes use of a hybrid DL model. In this work, a hybrid form of bidirectional long short-term memory (BLSTM) and enhanced recurrent neural network (ERNN) is developed to correctly label the features of the input datasets ^[20,21,22]. In the presented method, the features were passed initially to the BLSTM layer and later to the ERNN layer for precisely mapping the features to the accurate label. The BLSTM network makes use of forward and backward passes for learning the feature, while the ERNN makes use of feedback from context layer for accurately predicting labels. BLSTM is a kind of recurrent NN (RNN) with 2 LSTM units collectively compiled in distinct directions. The forward and backward layers of LSTM process the feature in forward and backward directions. These features are an enhancement of the conventional LSTM for preserving the previous and upcoming datasets. As well, this might assist in better understanding the context of classifying brain tumors. The LSTM unit encompasses forget, input, output and hidden gates in cell state, and the computation of LSTM forward units is defined as follows:

Now, ${\zeta }_{t}$ represents forget gate outcome; ${\eta }_{t}$ specifies input gate outcome; ${0}_{t}$ shows the total resultant of output gate; ${\xi }_{t}$ refers to the candidate cell states output; ${\varTheta }_{t}$ signifies the cell state output; ${\mathrm{ћ}}_{t}$ characterizes the hidden state outcome; $\sigma$ means the sigmoid activation function and $\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}$ activation functions; ${\mathrm{ћ}}_{t-1}$ stands for the prior hidden state outcome; ${\chi }_{t}$ denotes the input; ${\omega }_{\zeta },$ ${\omega }_{\eta },$ ${\omega }_{o}$ and ${\omega }_{\xi }$ symbolize the weight vectors of forgetting, input, output and cell state gates; ${\beta }_{\zeta }, {\beta }_{\eta }, {\beta }_{0}$ and ${\beta }_{\xi }$ indicate the bias vectors of corresponding gates; and ${\varTheta }_{t-1}$ is represents the outcome of earlier cell state. The computation is also considered for LSTM units in the backward direction. The total resultant of the hidden state in a backward direction is provided in the following expression:

The output of Bi‐LSTM layer was transferred to the ERNN layer for appropriately labeling the feature. The major reason for the hybridization of BLSTM with the ERNN was to improve the classification performance. This can be accomplished by increasing the amount of data available for the NN in the training stage. The ERNN layer encompasses output, input and hidden components for classification and feature extraction. The input layer directly sends the input towards the hidden layer (HL), and the HL is accountable to derive features from input. One of the main advantages of the hybrid method is that the discriminatory nature of the method will be increased. Thus, the model can recognize the differences among the features and label them accordingly. The HL of ERNN is defined by Eq (12):

Now, ${A}_{t}$ represents the output of HL, ${f}_{\lambda }$ indicates activation function, ${\chi }_{t}$ denotes the input to ERNN, ${\varpi }_{\lambda }$ and ${v}_{\lambda }$ represent the weight matrix, and ${\beta }_{\lambda }$ represents bias vector. The ERNN method encompasses a context layer including self‐connected feedback for providing previous data. The context layer is defined below:

In Eq (13), ${c}_{t-1}^{1}$ indicates the output of $l$ - $th$ context layer. ${A}_{t}^{j}$ represents output of the ${j}^{th}$ HL, and it is evaluated as follows:

In Eq (14), ${f}_{0}$ denotes activation function of the output layer, ${\varpi }_{O}$ indicates weight values of the output layer, and ${\beta }_{0}$ indicates bias vector of the output layer. The output from the FC layer was given to the $softmax$ layer in which probability is allocated to all classes. The probability dominance might assist in deciding the accurate class for the input feature.

To adjust the hyperparameter values of the hybrid DL model, the Nadam optimizer is employed in this work. The Nadam optimizer tries to join Nesterov accelerated adaptive moment prediction with the Adam optimizer ^[23,24]. The advantage of this technique is that the used adaptive moment prediction helps to accomplish a very accurate stage in the gradient direction by updating module variables including the momentum stage before the gradient computation ^[25,26]. The upgrade rule of Nadam is formulated as

where

3.   Results

In this section, the activity recognition of the IWSODL-MAHAR method is tested by utilizing the UCI HAR dataset ^[27]. It is a balanced dataset which comprises 10299 samples with six class labels as portrayed in Table 1. The sample images are shown in Figure 2.

The confusion matrices of the IWSODL-MAHAR model are shown in Figure 3. The figure represents that the IWSODL-MAHAR method has gained effectual recognition of human activities under all aspects.

Table 2 exhibits the overall activity recognition outcome of the IWSODL-MAHAR model with 60% of Training (TR) databases and 40% of Testing (TS) databases. In Figure 4, the overall HAR outcome of the IWSODL-MAHAR method on 60% of TR database is given. The experimental values demonstrated that the IWSODL-MAHAR model identified all activities. It is observable that the IWSODL-MAHAR model attained average $acc{u}_{y}$ of 99.47%, $sen{s}_{y}$ of 98.40%, $spe{c}_{y}$ of 99.68%, ${F}_{score}$ of 98.39%, receiver operating characteristic (ROC) $RO{C}_{score}$ of 99.04% and Matthews correlation coefficient (MCC) of 98.07%.

In Figure 5, an overall HAR outcome of the IWSODL-MAHAR approach on 40% of TS databases is given. The experimental values exhibited that the IWSODL-MAHAR system has identified all activities. It can be apparent that the IWSODL-MAHAR system has reached average $acc{u}_{y}$ of 99.39%, $sen{s}_{y}$ of 98.14%, $spe{c}_{y}$ of 99.63%, ${F}_{score}$ of 98.18%, $RO{C}_{score}$ of 98.89% and MCC of 97.81%.

Table 3 displays the overall activity recognition outcome of the IWSODL-MAHAR approach on 70% of TR and 30% of TS databases. In Figure 6, an overall HAR outcome of the IWSODL-MAHAR approach on 70% of TR databases is given. It can be obvious that the IWSODL-MAHAR system has accomplished average $acc{u}_{y}$ of 99.55%, $sen{s}_{y}$ of 98.59%, $spe{c}_{y}$ of 99.73%, ${F}_{score}$ of 98.60%, $RO{C}_{score}$ of 99.16% and MCC of 98.33%.

In Figure 7, an overall HAR outcome of the IWSODL-MAHAR system on 30% of TS database is given. The experimental values exhibited that IWSODL-MAHAR algorithm has identified all activities. It can be apparent that the IWSODL-MAHAR algorithm has achieved average $acc{u}_{y}$ of 99.59%, $sen{s}_{y}$ of 98.71%, $spe{c}_{y}$ of 99.75%, ${F}_{score}$ of 98.74%, $RO{C}_{score}$ of 99.23% and MCC of 98.50%.

4.   Discussions

The IWSODL-MAHAR algorithm's training accuracy (TACC) and validation accuracy (VACC) are assessed using HAR performance in Figure 8. The figure shows that the IWSODL-MAHAR algorithm performed better at maximum TACC and VACC values. The IWSODL-MAHAR algorithm has evidently produced better VACC results.

In Figure 9, the IWSODL-MAHAR system's training loss (TLS) and validation loss (VLS) are evaluated in relation to HAR performance. The graphic shows that the IWSODL-MAHAR algorithm performed better with low TLS and VLS values. The IWSODL-MAHAR system has produced less favorable VLS outcomes, as may be seen.

Figure 10 details an evident precision-recall analysis of the IWSODL-MAHAR system in the test database. The IWSODL-MAHAR approach produced higher precision recall values in many classes, as shown by the figure. Finally, the detailed comparative analysis of the IWSODL-MAHAR with recent methods is given in Table 4. Figure 11 offers a comparative $acc{u}_{y}$ study of the IWSODL-MAHAR. The results implied the enhanced performance of the IWSODL-MAHAR model. Based on $acc{u}_{y}$, the IWSODL-MAHAR has reached an improved value of 99.59%. Contrastingly, the CNN ^[28], ensemble AE ^[29], Support Vector Machine (SVM) ^[30], ConvAE-LSTM ^[31], RelieF and hybrid gradient based optimizer- grey wolf optimizer feature selection (GBO-GWO) models have reported reduced $acc{u}_{y}$ of 96.88%, 80.80%, 91.12%, and 98.67%. 96.71%, and 98.52%, respectively.

Figure 12 provides a comparative $sen{s}_{y}$ and $spe{c}_{y}$ analysis of the IWSODL-MAHAR technique. The outcomes showed the higher performance of the IWSODL-MAHAR approach. With respect to $sen{s}_{y}$, the IWSODL-MAHAR system has gained an enhanced value of 98.71%. Contrastingly, the CNN, ensemble AE, SVM, ConvAE-LSTM, RelieF and hybrid GBO-GWO methods have reported minimal $sen{s}_{y}$ of 97.64%, 81.36%, 91.75%, 98.27%, 97.49%, and 97.90%, respectively.

Along with that, in terms of $spe{c}_{y}$, the IWSODL-MAHAR has reached an improved value of 99.75%. Contrastingly, the CNN, ensemble AE, SVM, ConvAE-LSTM, RelieF and hybrid GBO-GWO approaches have reported minimal $spe{c}_{y}$ of 98.19%, 82%, 92.38%, 98.91%, 98.14% and 98.11%, respectively. The improved performance of the IWSODL-MAHAR technique over other current models was highlighted by these results.

5.   Conclusions

In this paper, an automated IWSODL-MAHAR method has been presented for movement and action detection. The major intention of the IWSODL-MAHAR algorithm is to recognize various kinds of human activities. Initially, data preprocessing is performed to make the input activity data a compatible format for the activity recognition process. In addition, the IWSO algorithm is applied as a dimensionality reduction technique. For activity recognition, the IWSODL-MAHAR technique has employed a Nadam optimizer with a hybrid DL model. The Nadam optimizer is applied as a hyperparameter tuning technique to boost the classifier results. The experimental evaluation of the IWSODL-MAHAR approach is assessed on benchmark activity recognition datasets. The experimental outcomes outlined the supremacy of the IWSODL-MAHAR methodology compared to recent models. In the future, an ensemble of DL oriented fusion methods will be designed to boost the recognition performance.

Acknowledgments

This research has been funded by the Deanship for Research & Innovation, Ministry of Education in Saudi Arabia, funding this research work through project number IFP22UQU4281768DSR122.

Authors' contributions

Conceptualization, YA, SR. Methodology, TT, KN. Software, validation and formal analysis, YA, SR. Investigation, YA, KN. Resources, TT, SR. Data curation, YA. Writing—original draft preparation, YA, SR. Writing—review and editing, TT, KN. Visualization, KN, SR. Supervision, YA. Project administration, TT.

Conflict of interest

All authors declare no conflicts of interest in this paper