Remaining useful life prediction of the lithium-ion battery based on CNN-LSTM fusion model and grey relational analysis

Abbreviations: 1D: One-Dimensional; 2D: Two-Dimensional; 3D: Three-Dimensional; ARIMA: Autoregressive Integrated Moving Average model; Adam: Adaptive Moment Estimation; BP: Back Propogation; CC: Constant Current; CNN: Convolutional Neural Network; CV: Constant Voltage; EOL: End of Life; HI: Health Index; LSTM: Long Short-Term Memory; MAPE: Mean Absolute Percentage Error; MSE: Mean Square Error; NASA: National Aeronautics and Space Administration; PoF: Physics of Failure; PSO: Particle Swarm Optimization; RUL: Remaining Useful Life; RVM: Ruby Version Manager; SOC: State of Charge; SOH: State of Health; SVR: Support Vector Regression

Nomenclature: ${C}_{N}$: Current battery capacity; ${C}_{R}$: Factory rated capacity of the battery; $\stackrel{-}{V}$: Average measured voltage; $RU{L}_{error}$: Remaining useful life prediction error; $RU{L}_{pr}$: Remaining useful life prediction results; $RU{L}_{tr}$: Remaining useful life actual results

1.   Introduction

With the emergence of the traditional energy crisis, there is a pressing for society to explore and develop new energy. Among many new energy sources, lithium-ion batteries have become the mainstream of the new energy market owing to their high energy density, high output voltage, long cycle life, and wide operating temperature range ^[1,2,3]. However, the internal resistance increases with lithium-ion batteries' repeated charging and discharging cycles. After the internal resistance increases, the battery heats severely, which continue to affect the performance and normal use of the battery pack ^[4,5]. The remaining useful life (RUL) of lithium-ion batteries is the number of charging and discharging cycles remaining between the beginning of measurement and the end of life (EOL) of lithium-ion batteries ^[6]. Regular RUL prediction of a lithium-ion battery can show the remaining useful cycle times of the battery, predict whether the battery is close to the EOL, and avoid potential risks in the use process ^{[7,8,9,10,11]}. Therefore, the accuracy of the lithium-ion battery RUL evaluation method will directly affect the overall performance of the battery management system, which has great practical significance in the field of energy battery application.

The traditional life prediction model is a tedious and strict process due to the complex physical and chemical properties of lithium-ion batteries. Fortunately, the RUL model of lithium-ion battery based on data-driven technology is a powerful and effective method with the development of artificial intelligence. It regards the battery as a black box, bypasses the complex change process inside it, and only needs to find the statistical law through the historical measurement data to predict the RUL of lithium-ion batteries. In recent years, more and more scholars have started to focus on the research of power batteries. There are two main categories for building battery life prediction models: model-based and data-driven methods ^[12,13,14].

The model-based method establishes the mathematical model of the battery by analyzing the physical structure and electrochemical reaction and then estimating the changing process of the battery parameters. Khare et al. ^[15] used the statistical modelling method to establish the mapping model between battery internal resistance and health state to evaluate the health state. Based on the analysis of the failure principle and the electrochemical reaction of lithium-ion batteries, a complete mathematical model was found to fit the degradation trajectory of lithium-ion batteries and to achieve the prediction of RUL. Mevawalla et al. ^[16] proposed an equivalent circuit model approach incorporating physio-chemical theory into developing a nonlinear equation for internal resistance. This method creates a model to simulate the internal resistance and surface temperature of lithium-ion batteries using actual measurable parameters. Wang et al. ^[17] proposed a resistance-based thermal model of the batteries considering the impact of the state of charge (SOC), battery temperature, and current on the battery heat generation. According to the research results, air velocity has little effect on the maximum battery temperature at the discharge rate of flying cars. However, it can affect the temperature decrease rate. Xie et al. ^[18] proposed a distributed spatial-temporal online correction algorithm for SOC-three-dimensional state of temperature (SOT) co-estimation of battery. The result shows that the co-estimation algorithm still has a good converge performance with disturbance added. Xing et al. ^[19] proposed a fusion prediction method based on the physics of failure (PoF) and data-driven technology. This method can deeply analyze the failure mechanism caused by changes of physical and chemical characteristics in the battery. It can also be applied to estimate some parameters in real-time monitoring scenarios. Wang et al. ^[20] introduced a spherical particle filter to solve the state space model and then established the state space model of battery capacity to predict the RUL of lithium-ion batteries after evaluating the capacity degradation. Tran et al. ^[21] investigated and compared the performance of three different equivalent circuit models using four lithium-ion battery chemistries under dynamic and non-dynamic current profiles. However, the model-based method is vulnerable to the influence of the external environment, and it is difficult to establish an accurate mechanism model. Also, due to the different physical and chemical properties of different batteries, the applicability of the model is not strong. It needs to be modified for different batteries, which requires much work.

The data-driven method uses techniques such as machine learning to extract battery ageing characteristics from battery data collected during operation, revealing the relationship between the input data and the system degradation process and thus predicting the remaining battery life ^[22,23,24]. Khumprom et al. ^[25] used multiple data-driven artificial intelligence algorithms to conduct a comparative experiment on predicting the remaining useful life of lithium-ion batteries. Cai et al. ^[26] proposed an optimization process based on a nondominated sorting genetic algorithm (NSGA II), which uses the short-term characteristics of support vector regression (SVR) and current pulse test to predict. Nevertheless, the model is relatively simple, and the potential is limited, which limits the prediction accuracy. Qin et al. ^[27] established an improved particle swarm optimization-support vector regression (PSO-SVR) model to estimate RUL under different fault thresholds. However, this method is challenging for dealing with massive data. Cai et al. ^[28] proposed a hybrid data-driven algorithm to predict the RUL of the lithium-ion battery, reconstructed the phase space, and established a model to predict the RUL by combining discrete grey model (DGM), relevance vector machine (RVM), and artificial fish swarm algorithm (AFSA). However, this method is effective in short-term prediction, and the long-term prediction error of the No. 7 battery is significant. Gou et al. ^[29] proposed a hybrid data-driven method integrating elm and random vector functional link (RVFL) networks to improve the generalization ability, accuracy, and robustness of prediction results. Ma et al. ^[30] and Zhang et al. ^[31] applied LSTM to predict the RUL of lithium-ion batteries. Yalçın et al. ^[32] proposed a novel scheme, namely CNN-artificial bee colony (ABC) leveraged from CNN and ABC algorithm for heat generation rate (HGR) and voltage estimation. Wang et al. ^[33] proposed a transferable lithium-ion battery RUL prediction method from cycle-consistency of degradation trend, which can solve both the problem of large dispersion of lifetime distribution and the issue of error accumulation. Wang et al. ^[34] proposed an improved feedforward-long short-term memory (FF-LSTM) modelling method to realize an accurate whole-life-cycle SOC prediction by effectively considering the current, voltage and temperature variations. Xia et al. ^[35] proposed a hybrid prediction model based on LSTM and a fully connected layer to capture the correlation in historical data. However, the above data-driven methods have the defects of complex structure, a large amount of calculation, and extended training time.

As mentioned earlier, with the progress of artificial intelligence algorithms, applying data-driven methods to power battery capacity prediction is a hot issue ^[36,37,38]. Due to the high dimensionality and non-linearity of the data collected during the discharge of lithium-ion batteries, the historical data features in complex non-linear systems can be extracted by the data-driven method ^[39,40,41]. However, single-model prediction methods still face the drawbacks of cumulative prediction errors and poor universality. The multi-model fusion prediction method can combine the advantages of each model and make up for the deficiency of single-model prediction. Therefore, the RUL prediction model for lithium-ion batteries composed of different in-depth learning techniques has excellent potential in prediction accuracy and generality. CNN can extract the features of one-dimensional (1D) time series data, mining potential hidden information, but it cannot learn the long-dependent characteristics of data. LSTM can solve the long dependence problem in time series data and effectively identify data patterns, but it is better than CNN in extracting data features ^[42,43]. A RUL prediction model for lithium-ion batteries based on the fusion of CNN and LSTM is presented to solve the RUL prediction problem. Combining the advantages of CNN and LSTM, extract the space-time characteristics of the lithium-ion batteries discharge process data to achieve RUL prediction for lithium-ion batteries. The main contributions of this study are summarized as follows.

1) Using grey relational analysis selected attributions affecting RUL of lithium-ion batteries, including mean discharge current, average discharge voltage, and average temperature as HI of batteries.

2) We proposed a RUL prediction of the lithium-ion battery based on CNN-LSTM fusion model. By changing the input data dimension and model hierarchy, the fusion model can capture the non-linear characteristics and time-space relationship of the discharge process of lithium-ion batteries.

3) By comparing BP, PSO-SVR, ARIMA-SVR, LSTM, and CNN-LSTM models, we found that the prediction accuracy of traditional machine learning models is not as good as that of the deep learning model, and the performance of our proposed CNN-LSTM fusion model is better than that of the single LSTM model. The MAPE and MSE of the proposed models are 0.36% and 0.38e-4, respectively.

4) Our study is different from the conventional RUL prediction. Considering the relaxation effect of the battery, we choose the last intersection of the battery capacity and the failure reference line as the failure threshold of the battery life. In the comparison experiments, the CNN-LSTM fusion model has the highest accuracy in predicting the RUL of lithium-ion batteries.

The rest of this paper is organized as follows. Section 2 describes the RUL prediction problem of lithium-ion batteries and the data structure. Then, in Section 3, the details of the proposed approach are introduced. Then, in Section 4, the proposed model is compared with the backpropagation (BP) model, long short-term memory (LSTM) model, particle swarm optimization hyphen support vector regression model (PSO-SVR) ^[25], and the autoregressive integrated moving average hyphen support vector regression model (ARIMA-PSO) ^[44], and the all-around performance of each model in the RUL prediction experiment are analyzed. Finally, the conclusion is presented in Section 5.

2.   Problem statement of RUL prediction of lithium-ion batteries

2.1.   The problem of RUL prediction of lithium-ion batteries

Most current new energy vehicles, communication devices, and electronic devices in the 5G smart era use lithium-ion batteries as their power support ^[45]. Usually, a single lithium-ion battery consists of five parts: a positive electrode, diaphragm, negative electrode, organic electrolyte, and battery shell ^[46]. The lithium-ion battery structure schematic diagram is shown in Figure 1.

The lithium-ion battery is a rechargeable battery which mainly relies on the movement of lithium-ion between the positive and negative electrodes to work ^[47]. During the discharge process of the lithium-ion battery, the electron (e-) and lithium-ion (Li+) simultaneously move. The e- runs from the negative electrode through the external circuit conductor to the positive electrode. The Li+ comes out of the negative electrode into the electrolyte and to the positive electrode through the diaphragm to combine with an electron. During the charging process, the Li+ comes out of the positive electrode into the electrolyte cell, shuttles from the diaphragm to the negative electrode, and embeds itself in the micropores of the carbon layer. Lithium-ion batteries are reversible in the chemical reactions that occur in the charging or power generation state and are used repeatedly by cyclic charging and discharging ^[48]. The working principle of lithium-ion batteries is shown in Figure 2.

However, during lithium-ion batteries' charging and discharging process, some undesired side reactions occur between the positive and negative electrode materials and the elements in the electrolyte. The side reactions generate a large amount of polymer accumulation around the positive and negative electrodes or the diaphragm, which prevents Li+ from deem bedding between the positive and negative electrodes, and shuttling in the electrolyte is the cause of lithium-ion battery performance degradation. So, with the increase in lithium-ion battery usage time, over-charging or over-discharging, the battery range will gradually decay, and the battery life will be shortened, leading to battery failure when the life threshold is reached. Therefore, the RUL prediction of lithium-ion batteries can ensure the safety and reliability of the battery use process and avoid risk.

The RUL of a lithium-ion battery is the number of charge/discharge cycles remaining between the start of the measurement and the threshold of failure, and its calculation formula is obtained by Eq (1).

where, $Cycle$ is the charge/discharge cycles of the lithium-ion battery at the measurement moment, and ${Cycle}_{EOL}$ is the charge/discharge cycles of the lithium-ion battery at the failure threshold. The lithium-ion battery degradation to a certain level will affect normal use, and 70% of the standard capacity of lithium-ion batteries is usually used as the failure threshold in research ^[49].

The RUL is based on the state of health (SOH) of lithium-ion batteries, and the RUL evaluation of the battery is performed after establishing the capacity degradation model of the battery. Among them, the SOH of a lithium-ion battery is the ratio between the remaining capacity and the initial capacity. The remaining capacity is the discharge capacity of the battery at this stage after it is fully discharged under standard discharge conditions. The percentage of these two capacities reflects the SOH of the battery at this stage. The most intuitive expression of the battery SOH is the degree of capacity decline, with the capacity of the battery in two stages as the main component, and its mathematical expression is obtained by Eq (2).

where, ${C}_{N}$ is the current battery capacity, ${C}_{R}$ is the factory rated capacity of the battery, and SOH expresses the current health of the battery in the form of a percentage of capacity. The most common way to measure SOH nowadays is this definition by capacity recession.

The lithium-ion battery capacity is used as the performance index to realize the RUL prediction. Still, the battery capacity is difficult to measure directly. Therefore, HI, which is highly correlated with the capacity of lithium-ion batteries, such as voltage, current, and temperature during battery operation, is extracted. Use the HI to construct the battery capacity prediction model to realize RUL prediction. The battery capacity is replaced when it is lower than a certain threshold, effectively eliminating the hidden danger of lithium-ion batteries.

2.2.   Data set

This paper uses the lithium-ion battery data set from NASA Ames Prognostics Center of Excellence ^[49]. The LiCoO2 is used as the positive material, soft and hard carbon as the negative material, and lithium salt as the electrolyte material for the 18,650 lithium-ion cobalt acid battery. The battery has a rated capacity of 2 Ah, and a rated voltage of 4.2 V. Lithium-ion batteries are charged (C-rate = 0.75 C), discharged (C-rate = 1 C) and tested for impedance at different temperatures until the end of the battery life. Record collected data, such as voltage, current, temperature, and impedance. Table 1 shows the details of the NASA battery pack.

We take the first group of lithium-ion batteries as an example to introduce the process of the NASA battery pack ageing life test. Charging process: charging with a constant current (CC) mode of 1.5 A until the voltage reaches 4.2 V. Then, charging continues with a constant voltage (CV) mode until the charging current drops to 20 mA. Discharge process: discharge with a CC of 2 A until the voltage of the battery reaches a different set value. For impedance measurements, the battery is scanned by electrochemical impedance spectroscopy (EIS) from 0.1 Hz to 5 kHz.

The condition for the end of battery life is that after recharging and discharging the lithium-ion battery repeatedly, the battery is considered invalid when its rated capacity decreases from 100% to 70% (from 2 Ah to 1.4 Ah). EOL of lithium-ion batteries can be defined as the number of cycles when the capacity of the lithium-ion batteries drops to the failure threshold during the initial experiment.

As can be seen from Table 1, the four batteries in the first group conformed to the standard charge/discharge cycle ageing experimental environment, with a long charge/discharge cycle number and apparent characteristics of capacity degradation. The second and third groups of batteries only changed the ambient temperature or discharge current, and the number of charge/discharge cycles was too short. The second group only had 28 cycles and did not reach the failure threshold. The cycle number of the third group of batteries is only 47, and it still needs to reach the failure threshold. Therefore, we selected the first group of batteries (B0005, B0006, and B0007 with 168 charge/discharge cycles) for ageing study and analysis.

3.   The RUL prediction model based on CNN and LSTM

3.1.   The advantage of CNN and LSTM

3.1.1.   Convolutional neural network (CNN)

Convolutional neural network (CNN) is an effective method to capture the characteristic information of data. CNN has an input layer, convolution layer, pooling layer, fully connected layer, an output layer and other structures ^[50]. Among them, the convolution layer and pooling layer are the core layers of CNN, which are the most representative. The most popular use of CNN is the feature extraction of high-dimensional image data by two-dimensional (2D)-CNN or three-dimensional (3D)-CNN in image processing. Recently, researchers have gradually applied CNNs in the study of feature extraction from 1D multivariate data ^[51].

The convolution layer convolutes the input data with the convolution core to extract the potential features of the data. The fixed-size convolution core scans the whole data domain like human eyes. Multiple convolution cores with different weights evaluate and extract the different features of all aspects of the data through convolution operation. The specific operations of the convolution layer are obtained by Eq (3).

where, ${W}_{i}^{l}$ is the $ith$ convolution kernel weight matrix in the $lth$ layer; ${X}^{l-1}$ is the output in the $lth$ layer; ${x}_{i}^{l}$ is the $ith$ feature of output in the $lth$ layer; "*" is a convolution operator; ${b}_{i}^{l}$ is the offset term. In this paper, the rectifier linear unit (ReLU) is selected as the activation function of the convolution layer, and its expression is obtained by Eq (4).

The high-dimensional image data in image processing contains a more significant amount of information. Multidimensional convolution neural network is mainly used, while the lithium-ion battery data is 1D time series data. So 1D-CNN is designed to extract the features of the battery data. The structure of the 1D convolution layer is shown in Figure 3.

After feature extraction of the convolution layer, multiple feature matrices are obtained according to different convolution cores. In order to extract enough hidden information, the output dimension of the convolution layer is generally significant. The role of the pooled layer is to down sample input features, filter many features simultaneously, and enhance some significant features, equivalent to the practical information filtering of human vision for observing things. The operator of the pooled layer is called the pooled kernel, which scans the characteristic matrix of the convolution layer output transversally by Eq (5).

where, ${x}_{i}^{l}\left(k\right)$ is the element of the $ith$ characteristic matrix of the $lth$ layer in the pooled core region; ${y}_{i}^{l+1}\left(j\right)$ is the element in the $ith$ characteristic matrix of the $(l+1)th$ layer after pooling. ${D}_{j}$ is the area covered by the $mth$ pooled core.

3.1.2.   Long short-term memory (LSTM)

When it comes to the processing of time series data, the most commonly used is a recurrent neural network (RNN). However, when the valuable information in the processed data is far away from the location where the information is needed, RNN cannot learn the characteristic information of the data well. Long short-term memory network (LSTM) is a variant of RNNs designed to cope with these gradient varnishing problems ^[52].

Especially in the lithium-ion battery RUL prediction scenario, the particular hidden layer neurons of LSTM make it possible to record the battery history input information while influencing the data input at the next moment. Therefore, the battery ageing data is used to extract feature parameters to input into the LSTM network for training. LSTM introduces three "Gates" into neurons to control the amount of information retained by the data before and after to solve the problem of a medium and long-term dependence on data ^[53]. The three gates are the input gate, forget gate, and output gate. The cell structure of LSTM neurons is shown in Figure 4.

As shown in Figure 4, $\sigma$ is the sigmoid activation function, the output range is $\left(\mathrm{0, 1}\right)$, the output range of tanh is $(-\mathrm{1, 1})$, and the value is limited within a specific range. LSTM is a variant of RNN, which controls the state of neurons through the three gates shown in Figure 4.

Formally, the formulas to update an LSTM unit at time tare by Eqs (6)–(11).

where, $\sigma$ is the element-wise sigmoid function. ${x}_{t}$ is the input vector at time $t$, and ${h}_{t}$ is the hidden state (also called out-put) vector storing all the useful information at (and before) time $t$. ${W}_{f},$ ${W}_{i},$ ${W}_{C}, {W}_{o}$ are the weight matrices for hidden state ${h}_{t}$. ${b}_{f}, {b}_{i}, {b}_{c}, {b}_{o}$ denote the bias vectors.

3.2.   RUL prediction model based on CNN-LSTM

3.2.1.   Data preprocessing

1) Property means taking the average value instead

The values of each attribute in the dataset are collected moment-by-moment basis, and the number of features is too large to increase the burden on the model, so the average value is used as an equivalent replacement for a certain attribute. Take measuring voltage as an example, and the average value calculation is obtained by Eq (12).

where, ${V}_{i}$ is the measured voltage at time $i$, $n$ is the number of measurements, i.e., the number of time points, and $\stackrel{-}{V}$ is the average measured voltage. Similarly, the average current and average temperature are preprocessed in this way and converted into characteristic variables.

2) Extraction of important attributes by grey relational analysis

There are coupling relationships between attributes in the Li-ion battery dataset. If all the attributes are input into the model, this will increase the model calculation time and reduce the prediction accuracy. Therefore, it is necessary to filter out the attributes that greatly impact the battery RUL through grey correlation analysis. In this paper, the capacity sequence of the cell is used as a reference sequence for the behavioral characteristics of the reaction system $Y = \left\{y\right(k\left)\right|k = \mathrm{1, 2}, \mathrm{\Lambda }, n\}$, The time series of equal voltage drop discharges with discharge voltages in the interval from 3.8 to 3.5 V is treated as a comparative series consisting of factors affecting the behavior of the system ${X}_{i} = \left\{{x}_{i}\right(k\left)\right|k = \mathrm{1, 2}, \mathrm{\Lambda }, n\}$, $k$ denotes the length of the sequence, $i$ denotes the number of compared sequences, and the calculation of the grey scale factor is obtained by Eq (13).

The calculating of the correlation is obtained by Eq (14).

The correlation degree ${r}_{i}$ takes a value between (0, 1), and closer to 1 indicates that the reference sequence is more correlated with the comparison sequence. As shown in Table 2, the average discharge current, average discharge voltage and average temperature were selected as the feature variables for training the prediction model of the RUL of lithium-ion battery after the analysis of B0005 and B0006 model batteries.

3) Normalization process

The value domain of each feature is different, and the magnitude is different. For example, the average discharge current size is around 2A, while the average temperature is around 30℃. Suppose the corresponding data processing is not carried out. In that case, the model is more obviously affected by the temperature feature and even ignores the role of the current-voltage feature on it, which is not consistent with the model design idea, so the normalization process is carried out to eliminate the influence of different magnitudes on the model by Eq (15).

where, ${x}_{k}$ denotes the specific value under any attribute, ${x}_{max}$ denotes the maximum value of the attribute, and ${x}_{min}$ denotes the minimum value of the attribute. The normalization process shrinks the range of values of each attribute to between [0, 1], which can eliminate the influence of different magnitudes on the correlation degree.

4) Data dimensional transformation

The CNN-LSTM model requires appropriate preprocessing of the data to allow the model to be trained at a specific step to learn the features of the data.

Assuming the number of input samples is 'N', and the number of sample features is 'Feature', the original shape of the input data is (N, Feature). The sample data is divided into N-TimeStep+1 sample data by time step (TimeStep) through data preprocessing, and the data shape becomes (N-TimeStep+1, TimeStep, Feature). That is the shape of the data changes from 2D to 3D.

3.2.2.   Structure of the proposed CNN-LSTM model

In order to predict the RUL of lithium-ion batteries, it is necessary to analyze the battery discharge data in detail. Since CNN extracts potential hidden information from the data and LSTM can solve the long-term dependence of time series data, we combine the advantages of these two algorithms and apply to the study of RUL of lithium-ion batteries. By comparing the battery capacity decline curves at different temperatures, different discharge voltages and different discharge currents, we can obtain the external factors affecting the lifetime of lithium-ion batteries and find out the most relevant variables to the battery capacity decline. That is, the essential attributes are extracted from the many influencing factors, and a battery capacity decay model is established to achieve the prediction of RUL of lithium-ion batteries. To this end, a CNN-LSTM based fusion model is designed in this paper. It combines CNN and LSTM models to complement their advantages and accurately and effectively predict the RUL of lithium-ion batteries.

The Keras library is usually used in traditional research to stack CNN and LSTM models. This paper analyzes the model principle and improves the structure of the CNN-LSTM fusion model based on the principles of CNN and LSTM algorithms. A lithium-ion battery RUL prediction model based on CNN-LSTM is designed. The flow chart of the model is shown in Figure 5 ^[31].

As mentioned above, the data shape changes from 2D to 3D after preprocessing. In order to enhance the features of the sample attributes, it is necessary to make the convolutional layers in CNN-LSTM act on the features within each time step simultaneously. Therefore, the model uses the TimeDistribution layer in the Keras library to package 1D-CNN and MaxPooling and applies them to the changed time steps in the sample data, respectively. This process requires that the data processing be two-dimensional within each time step, using the 1D-CNN to slide over the data features, thus filtering and reinforcing the features. As in Figure 5, the row vector within the left time step enters the CNN processing stage and deforms into 2D data, changing the data shape again to split the time step into new time steps and subsequences (TimeStep', SubSequence) so that each new time step is 2D data (N-TimeStep+1, TimeStep', SubSequence, Feature). Then, the data features are extracted with 1D-CNN, the features are reinforced by the pooling layer, and then the output is tiled with Flatten layer to get the reinforced features, i.e., Feature'. At this point, the data shape becomes (N-TimeStep+1, TimeStep', Feature'). The same TimeDistribute layer is used to wrap the Dense layer to function simultaneously within all new time steps to speed up the convergence. Then, the temporal data output from the fully connected layer is fed to the LSTM layer for processing, and the neurons in the LSTM layer correspond to the changed time steps to finally output the prediction results.

The goal of deep learning is to fit the output by continuously changing the network parameters so that the parameters can do various nonlinear transformations on the input, which is essentially a function to find the optimal solution. Adaptive moment estimation (Adam) is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process by iteratively updating the neural network based on training data weights. The gradient calculation of the objective function is obtained by Eq (16).

where, ${g}_{t}$ is the gradient of the objective function to $\theta$. The average value at the first time is obtained by Eq (17).

where, ${\widehat{m}}_{t}$ is the average value of the gradient at the first time, ${\beta }_{1}$ is the decay factor of the first momentum. The second time variance is obtained by Eq (18).

where, ${\widehat{v}}_{t}$ is the second noncentral variance value of the gradient, ${\beta }_{2}$ is decay factor of infinity norm. The parameter update is obtained by Eq (19).

where, $\eta$ is the learning rate, $ϵ$ is the default parameter, avoiding zero denominator.

4.   Simulation and analysis of RUL prediction

4.1.   Experimental setting

The experiment selected B0005, B0006, and B0007 batteries, 168 cycles of charge and discharge, as the experimental subjects in this experiment. The failure reference line for lithium-ion batteries is 70% of the rated capacity. The RUL of a battery is the residual discharge cycle between the actual current capacity and the EOL. When the capacity decay curve drops to the failure reference line, it does not necessarily mean that the battery is no longer working. In fact, the battery can still be used after the rest time ^[54]. Due to the internal current flow when the battery is at rest, the active material is rebalanced, increasing the available capacity for the next cycle ^[55]. So, the EOL is not the first intersection of the battery capacity and the failure reference line but the last intersection. Therefore, the EOL chosen in this experiment is the latest intersection of the battery capacity and the failure reference line.

Attribute values in a dataset are collected in time units, with hundreds of points each time. Mean values are used as an equivalent substitute for an attribute because they are too large as features and add to the load on the model. The average discharge current, average discharge voltage and average temperature were selected as the input of the CNN-LSTM model, and the output was the capacity of the lithium-ion battery through grey relational analysis. The model selected the first 100 sets of data for training and the last 60 sets of data for testing.

Figure 6 shows the architecture of the CNN-LSTM prediction model. The step size and convolution kernel size of Conv1D are 1 and 2, respectively, and the number of filters is 10. The model uses the "same" mode to fill the data processed by the convolutional kernel, and the activation function is ReLU. The MaxPooling1D has a pool size of 3 and a step size of 1, selecting default mode, which means that the data entered into the layer is not populated. The size of Dense_1 is 5, and there are 200 neurons in the LSTM layer with a step length of 5. The initialization method of Dense_2 is "normal", and the size is 1. ReLU is also used for the activation functions of these three layers. The neural networks introduced in this study used MAE as the loss function during their training process. In this research, the Adam optimizer is employed to optimize the loss functions of the CNN–LSTM fusion model and the other benchmark models because of providing superior prediction performance (learning rate = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-8). The orange color at the bottom of the diagram indicates that the layer uses a TimeDistribute wrapper to act on the data. In the design of other models, the penalty factor C = 70 for PSO-SVR, the kernel function is RBF, gamma = 0.7, epsilon = 0.01, the number of particle swarms is 20, and the maximum evolutionary algebra is 100. The order p = 3, the difference number d = 1 and the sliding average model order q = 2 for the ARIMA-SVR model, the penalty factor C = 100, the kernel function is RBF, gamma = 0.7, epsilon = 0.01. The number of hidden layer neurons in the BP model is 12. The LSTM model has a time step of 10 and the first hidden layer (the number of LSTM cells) of 200. The second hidden layer is the Dropout layer, and the Dropout rate is set to 0.3; The third hidden layer is the fully connected layer, with one neuron number. Epoch and batch size of BP, LSTM and CNN-LSTM are set to 200 and 20, respectively.

4.2.   Evaluation indicators

In studies related to the SOH monitoring of lithium batteries, two evaluation metrics, prediction error of RUL $\left(RU{L}_{error}\right)$ and mean absolute percentage error (MAPE), are mainly used. Mean square error (MSE) is also introduced to make the model more accurate. Therefore, use these three evaluation indicators to better evaluate the predictive effect and reliability of the model.

$RU{L}_{error}$ is the error between predicted $RU{L}_{pr}$ and expected $RU{L}_{tr}$ of the model, which can be used as an evaluation index of model reliability. $RU{L}_{error}$ can be calculated by Eq (20).

MSE is the commonly used mean square error indicator, which is square and mean of errors for the battery capacity ${C}_{pr}$ and expected battery capacity ${C}_{tr}$. It can be used as an evaluation index of model accuracy and obtained by Eq (21).

MAPE is the mean of relative error of multiple prediction results of the model, which can be used as an evaluation index of model stability. MAPE can be calculated by Eq (22).

4.3.   Experiment results and analysis

The combined PSO-SVR ^[25] model of traditional machine learning algorithm SVR and group intelligence algorithm PSO, ARIMA-SVR model of SVR and time series prediction model ARIMA ^[44], BP network model and LSTM model of deep learning algorithm are designed experimentally. The prediction effect of the proposed CNN-LSTM fusion model is compared with the model mentioned above and two deep learning algorithms on three data sets. Figure 7 shows the performance of various prediction models in B0005, B0006, and B0007 data sets.

Since PSO is only a parameter optimization algorithm, the SVR mentioned in the experiment refers to the PSO-SVR algorithm. Figure 7 shows that the two models combined with SVR and the traditional machine learning algorithm in the three data sets have relatively poor prediction results. For example, in the B0006 data set, the BP model always deviates from the actual value in the late prediction period. In contrast, the LSTM and CNN-LSTM fusion models fit the test set well.

Specifically, each model learns more or less about the trend of residual capacity degradation. The traditional machine learning algorithm, SVR, cannot fit expected values well even though it optimizes parameters by the PSO particle swarm algorithm. Though the fusion model of SVR and ARIMA can predict the data development trend, there is some delay in the model, and the trend cannot be predicted in real-time. Meanwhile, the prediction results for the non-linear change process of the data could be better. That is, the fluctuation characteristics of the residual battery capacity due to the recovery effect are not learned. BP, LSTM and CNN-LSTM three neural network models have better fitting results than the two models of support vector machine. The BP neural network deviates in different degrees in the middle and late prediction and fits well in the B0005 data set. However, there is a specific deviation in the B0006 data set later prediction. Deviations occur on B0007 data sets from the mid-term and accumulate into large deviations at later stages, which do not converge to the failure threshold. LSTM and CNN-LSTM are better for long-term prediction than the BP network model and have better prediction stability for different datasets. LSTM and CNN-LSTM are better for long-term prediction than the BP network model and have better prediction stability for different data sets.

Figure 8 shows that the predicted lifetime endpoints of LSTM and CNN-LSTM in the B0005 dataset are almost identical to the actual values, approximately discharging in the 124th cycle. However, the BP network with a good fit falls near the 135 discharge cycles due to large fluctuations near the failure threshold. The RUL errors of SVR and ARIMA-SVR are more significant than those of the BP network. On the B0006 data set, the RUL predicted by SVR is quite different from the actual value, and the life endpoints predicted by LSTM and CNN-LSTM are still accurate. Surprisingly, ARIMA-SVR with poor fit is more accurate than the BP network in predicting EOL. It is known that if the capture requirements of data characteristics are not high and the accuracy of prediction is only required at a particular time, the ARIMA-SVR fusion model can be used, which also provides evidence for the feasibility of the fusion model. On the B0007 battery data set, ARIMA-SVR, LSTM, and CNN-LSTM are still the best predictors of remaining life. However, the BP network and SVR cannot converge near the failure threshold. Because the BP and SVR-related models need to be revised for long-term data learning during training. For this reason, the LSTM model enhances the learning of long-term data. So LSTM model and CNN-LSTM model can effectively fit the fluctuations of actual data in performance.

Figure 9(a) is the error map of each model predicting the B0005 data set. It can be seen more directly that the errors of LSTM and CNN-LSTM models fall to near zero with slight fluctuation. Figure 9(b), (c) is the error map of each model predicting the B0006 and B0007 data sets. From the figures, we can observe that the error fluctuations of LSTM and CNN-LSTM models are still small and fall near the error line of 0. The ARIMA-SVR model also performs well in both data sets with standard errors. Tables 3 and 4 contain the error indicators of each model in the B0005, B0006, and B0007 data sets, respectively. The minimum values have been bold. From Table 3, we can see that the mean square error MSE of the proposed CNN-LSTM model is the lowest, and the percentage relative error MAPE of the LSTM model is the lowest. It shows that the accuracy and stability of the LSTM model and the CNN-LSTM model for the B0005 dataset are better than the other models. Tables 4 and 5 show that the MAPE and MSE of the CNN-LSTM model are the minimum values in all models. It shows that the CNN-LSTM model performs best in both B0006 and B0007 data sets and that the CNN-LSTM model has the same RUL_error as LSTM in the B0005 data set, while the other two data sets are the smallest, indicating that the model has high reliability. Figure 10 shows how well each model predicts the failure threshold in each of the three battery data. The bars above the baseline indicate that the model delays predicting the failure threshold, and the bars below the baseline indicate that the model predicts the failure threshold early. As seen from the figure, in the B0005 and B0006 datasets, the BP model, PSO-SVR model and ARIMA-SVR have a large delay in predicting the failure threshold, and the LSTM and CNN-SLTM models have a better prediction. In the B0007 dataset, the BP model and PSO-SVR model could not predict the failure threshold. In contrast, the ARIMA-SVR, LSTM, and CNN-SLTM models could predict the battery failure threshold slightly earlier than predicted, with the proposed model accurately predicting the failure threshold. The CNN-LSTM fusion model has the best performance in predicting the battery RUL.

5.   Conclusions and future work

In this paper, we combine the advantages of CNN and LSTM and propose a fusion model based on CNN and LSTM for RUL prediction of lithium batteries. First, the main factors of the battery that affect the RUL degradation are screened as the HI of the battery using grey relational analysis. Then the data are processed in a specific way to extract the features of the 1D lithium-ion battery data using TimeDistribute wrapping CNN layer for each time step. In addition, the data was entered into the LSTM layer through a fully connected layer wrapped in TimeDistribute to analyze the long-term changes in the battery data and build a RUL prediction model for lithium-ion batteries. The proposed model has been experimented on the NASA lithium-ion batteries dataset compared to the traditional machine and single deep learning models. The experimental results show that the proposed CNN-LSTM fusion model can effectively monitor the capacity degradation process of lithium-ion batteries and can accurately predict the failure threshold of the RUL for batteries, considering the battery relaxation effect. In addition, the CNN-LSTM model shows the most robust performance in MAPE, MSE and RUL_error compared with the benchmark model, verifying that the deep learning model outperforms the machine learning model.

In general, the multi-model fusion approach is superior to a single model. Our proposed RUL prediction model for lithium-ion batteries can effectively predict the current battery capacity and avoid safety hazards caused by battery aging in practice. In addition, the final battery life is predicted based on the battery relaxation effect, which helps users to correctly understand the RUL of the battery and save the usage cost.

Although the model achieves good prediction accuracy, it still has shortcomings. For example, the effect of ambient temperature on the battery RUL prediction is not considered in the experiment, and the subsequent experiments of ambient temperature can be added for analysis. In addition, from the perspective of computational efficiency, increasing the number of parameters improves model accuracy but increases calculation time.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 61976055), Special Fund for Education and Scientific Research of Fujian Provincial Department of Finance (No. GY-Z21001), and Open Project Support of State Key Laboratory for Management and Control of Complex Systems (No. 20210116).

Conflict of interest

The authors declare there is no conflict of interest.