Intelligent body behavior feature extraction based on convolution neural network in patients with craniocerebral injury

1.   Introduction

With the continuous development of society, health is more and more important to people. With the continuous application of information technology in the medical field, people put forward the concepts of medical big data, intelligent nursing, intelligent hospital and so on. Medical care is an indispensable part of hospital work, and plays an important role in the process of patients' rehabilitation ^[1,2,3]. Now, the medical and nursing work of the hospital is mainly done manually, which is a very large workload. Therefore, the development of intelligent medical care, through the manufacture of nursing robots to assist nursing workers to complete nursing work, can not only reduce the workload of nursing workers, but also save costs for the hospital.

The application of convolution neural network based patient posture behavior pattern recognition technology to intelligent medical nursing can greatly improve the nursing level of intelligent medical nursing robot ^[4,5]. In the intelligent medical nursing, the nursing robot can detect the patient's posture and behavior, so as to judge the patient's intention and assist the patient who is physically inconvenient to complete some actions; The nursing robot can also monitor the patient's state. If the patient falls or other critical situation, it can give an alarm in time to inform the nursing staff; The nursing robot is combined with the body movement recognition technology of patients to design a more comfortable human-computer interaction mode and give patients a more humanized experience.

The patients with craniocerebral injury are unstable, slow recovery, and need a long time of nursing. Intelligent medical treatment based on convolutional neural network can detect the body behavior of patients with craniocerebral injury by extracting the characteristics of their body behavior, and make targeted response measures to care for the patients with brain injury ^[6,7,8]. Yolo algorithm is a target detection algorithm based on deep learning, it can quickly, efficiently, accurately and real-time detect the position of the target's bounding box both from human and animal ^[9]. In 2020, Sivamani et al. ^[10] obtained real-time posture detection of pigs using the Yolo algorithm. Thus, this paper proposes a method of extracting the patient's body behavior feature based on convolution neural network, in order to reduce nursing workload and save hospital costs.

2.   Materials and methods

2.1.   Research object

A total of 80 patients with craniocerebral injury in our hospital from January 2019 to April 2020 were selected. There were 47 males and 33 female, aged 19-67 (45.4 ± 12.7) years old. The causes of injuries were traffic accidents (51 cases), falling injuries (17 cases) and violent injuries (12 cases).

2.2.   Algorithm design

2.2.1.   Existing problems

There are three main problems in the recognition technology of patients' body behavior based on convolution neural network. Firstly, the individual differences and non rigid deformation of patients bring great difficulty to the detection of body behavior. Secondly, multi person scene and complex background will also affect the detection, so the real-time and reliability of the detection and recognition system can not be guaranteed. Finally, there is the diversity of perspectives, the diversity of human posture, and the different shooting perspectives of monitoring probes, which also brings great challenges to patient detection.

2.2.2.   Convolutional neural network

Convolution neural network is a kind of feedforward neural network, which is used to process the data with grid structure ^[11]. In order to solve the goal of detecting patients' body behavior in intelligent medical care, this paper designs the convolutional neural network structure and training tag, and proposes an algorithm to detect patients' body behavior. Finally, the proposed algorithm is verified and analyzed.

Convolutional neural network is different from traditional neural network. Convolutional neural network is generally composed of input layer, convolution layer, excitation layer, pooling layer and full connection layer. The input layer is mainly used to preprocess the input data, including de averaging, normalization and other methods. Convolution layer is the core of convolution neural network, which extracts features on the input layer. The excitation layer makes nonlinear mapping of the output of the convolution layer. Pooling layer is mainly responsible for compressing data and parameters to avoid over fitting. The full connection layer is located at the tail of convolutional neural network, which is the same as the traditional neural network ^[12,13].

In the research of intelligent medical nursing based on convolution neural network, this paper adopts double network model design, first designs the patient detection network model, and then designs the patient body movement feature extraction network model ^[14,15,16].

2.2.3.   Patient detection network

Patient detection is a kind of detection mechanism that uses a certain target detection algorithm to determine the target position. In this paper, the combination of target detection and patient body behavior feature extraction can better lock the target patient, and then recognize the patient's body behavior feature ^[17,18].

In the Yolo algorithm, the input image is divided into n × n square grids to determine whether the center of the target object falls inside the grid. If there is, mark the grid to represent that the grid can detect the target object. For each cell, m bounding boxes can be predicted and the confidence of bounding boxes is as follows:

where $\Pr (\mathit{object}{\rm{ }})$ represents the probability that the bounding box contains the detected target, and ${\rm{IOU}}_{{\rm{pred }}}^{{\rm{truth }}}$ represents the accuracy of the bounding box.

Use (x, y, w, h) to represent the size and position of the bounding box, where (x, y) represents the center coordinates of the bounding box, and W and H represent the width and height of the bounding box. Therefore, the predicted value of the bounding box is (x, y, w, h, c).

For each grid, K class probability values are given. This probability value is the conditional probability under $\Pr (\mathit{object})$, denoted as $\Pr \left({{\rm{clas}}{{\rm{s}}_{\rm{i}}}\mid {\rm{ object}}} \right)$. The confidence of bounding box category is defined as:

According to the above analysis, a 448 × 448 pixel image (input) was transferred to a vector representing the category, size and position of bounding box (output) by the Yolo algorithm. The key steps was as in Table 1.

In this study, the Yolo algorithm refers to the GoogleNet network design model, and uses the idea of induction module to design the patient detection network model. The patient detection network model is shown in Figure 1. It can be seen from the figure that the detection network structure includes an input layer, input 448 × 448 pixel image, 24 convolution layers, convolution core is mainly 3 × 3 and 1 × 1, 4 pooling layers, pooling window is 2 × 2, step size is 2, two full connection layers, one output layer, output layer outputs 7 × 7 × 30 vector.

2.2.4.   Body movement feature extraction network

The patient is detected and located by using Yolo algorithm, and then the patient's body movement feature extraction network model is used to extract the patient's feature information ^[9,19,20]. In this paper, three convolution layers with different convolution kernel sizes are proposed to simultaneously extract the body behavior characteristics of patients.

To solve problems in the recognition technology of patients' body behavior, this study improved the basic unit of perception in GoogLeNet model, and used three convolution layers with different convolution kernel sizes to extract the characteristics of patient's body behavior. The improved model is shown in Figure 2.

Three convolution layers with different convolution kernel sizes are used to extract features, which improves the accuracy of sign extraction and multi view.

After the input image is input through the input layer, three convolution layers with different convolution kernel sizes are used to extract features, and the output of the three convolution layers is connected through the CONCAT function to get a new feature map. In this paper, convolution kernels of 7 × 7, 5 × 5 and 3 × 3 are used. The body feature extraction network model is shown in Figure 3.

In the network structure model of body feature extraction, 30 new feature maps were obtained from the input data through the first perception structure; 90 and 270 feature maps are obtained through the second and third perception structures respectively; then the body behavior features are extracted through the convolution layer of two 5 × 5 convolution cores; finally, the output is through the convolution layer of 1 × 1 convolution core.

2.3.   Classifier design

After the network structure is designed, a classifier is designed for the patient's body behavior algorithm. After the feature extraction of the patient's body behavior, the recognition result of the patient's body behavior category is given according to the classifier ^[21,22].

In this paper, the global average pooling layer and softmax layer are used as classifiers. In this way, the network parameters of the identification system can be reduced, and the efficient operation of the system can be ensured. The classifier of patient body behavior recognition system is shown in Figure 4.

The main function of global average pooling layer is to reduce the dimension of feature map, and the main task of softmax layer is to output the final result by probability.

3.   Results and discussion

Firstly, the open source framework platform DARKNET and CAFFE are built for patient detection training and patient behavior feature extraction. Then the intelligent medical nursing system based on convolution neural network is tested. In the process of testing, 100,000 images of patient detection data set and 50,000 images of body behavior feature extraction network data set are prepared. Finally, the two data sets are divided into training set and test set, which are run on two framework platforms. Figure 5 shows the accuracy of the patient body behavior feature extraction network.

By increasing the number of test images, we can enrich the extracted features, and then screen out the features that can better represent the main content of the patient's body behavior. In this case, the accuracy of detecting the patient's body behavior feature extraction network will also rise. Based on about 48,000 images, the test accuracy will rise to 0.982. Table 2 shows the recognition rate of a certain action and the average recognition rate of all actions in the patient posture behavior recognition system. The average recognition rate of 97.8% shows that the convolution neural network can recognize the body behavior of patients better, and can give patients better nursing work on this basis.

4.   Conclusions

This study mainly studies the application of convolution neural network in image recognition, which connects image recognition with intelligent medical care, and proposes an intelligent medical care system based on convolution neural network, which provides reference and experience for the research of hospital intelligent medical care.

Acknowledgments

We would like to thank the patients and their families for their cooperation in the process of data collection.

Conflict of interest

All authors declare no conflicts of interest in this paper.