A new ensemble residual convolutional neural network for remaining useful life estimation

Long Wen; Yan Dong; Liang Gao; Long Wen; Yan Dong; Liang Gao

doi:10.3934/mbe.2019040

Mathematical Biosciences and Engineering

2019, Volume 16, Issue 2: 862-880. doi: 10.3934/mbe.2019040

Previous Article Next Article

Research article Special Issues

A new ensemble residual convolutional neural network for remaining useful life estimation

1.
The State Key Laboratory of Digital Manufacturing Equipment & Technology, School of Mechanical Science & Engineering, Huazhong University of Science & Technology, Wuhan, 430074, China
2.
School of Electronic Information & Communications, Huazhong University of Science & Technology, Wuhan, 430074, China

Received: 18 October 2018 Accepted: 10 December 2018 Published: 28 January 2019

Remaining useful life (RUL) estimation is one of the most important component in prognostic health management (PHM) system in modern industry. It defined as the length from the current time to the end of the useful life. With the rapid development of the smart manufacturing, the data-driven RUL approaches have been widely investigated in both academic and engineering fields. Deep learning, which is a new paradigm in machine learning, has been applied in the RUL related fields, and has achieved remarkable results. However, classical deep learning algorithms also encounter the vanishing/exploding gradient problem found in artificial neural network with gradient-based learning methods and backpropagation. In this research, a new residual convolutional neural network (ResCNN) is proposed. ResCNN applies the residual block which skips several blocks of convolutional layers by using shortcut connections, and can help to overcome vanishing/exploding gradient problem. What's more, the ResCNN is enhanced by using the k-fold ensemble method. The proposed ensemble ResCNN is conducted on the C-MAPSS data provided by NASA. The results show that the proposed ensemble ResCNN has achieved significant improvement in both the mean and the standard deviation of the prediction RUL values. The proposed ensemble ResCNN has also compared with other famous machine learning and deep learning methods, including Multilayer Perceptron, Support Vector Machines, Deep Belief Networks, Long Short-Term Memory Model, Convolutional Neural Network and many other methods in literatures. The comparison results show that ensemble ResCNN achieved the start-of-the-art results, and outperform almost all of them.
- residual network,
- convolutional neural network,
- ensemble learning,
- remaining useful life estimation,
- prognostic health management
Citation: Long Wen, Yan Dong, Liang Gao. A new ensemble residual convolutional neural network for remaining useful life estimation[J]. Mathematical Biosciences and Engineering, 2019, 16(2): 862-880. doi: 10.3934/mbe.2019040

Related Papers:

Abstract

Remaining useful life (RUL) estimation is one of the most important component in prognostic health management (PHM) system in modern industry. It defined as the length from the current time to the end of the useful life. With the rapid development of the smart manufacturing, the data-driven RUL approaches have been widely investigated in both academic and engineering fields. Deep learning, which is a new paradigm in machine learning, has been applied in the RUL related fields, and has achieved remarkable results. However, classical deep learning algorithms also encounter the vanishing/exploding gradient problem found in artificial neural network with gradient-based learning methods and backpropagation. In this research, a new residual convolutional neural network (ResCNN) is proposed. ResCNN applies the residual block which skips several blocks of convolutional layers by using shortcut connections, and can help to overcome vanishing/exploding gradient problem. What's more, the ResCNN is enhanced by using the k-fold ensemble method. The proposed ensemble ResCNN is conducted on the C-MAPSS data provided by NASA. The results show that the proposed ensemble ResCNN has achieved significant improvement in both the mean and the standard deviation of the prediction RUL values. The proposed ensemble ResCNN has also compared with other famous machine learning and deep learning methods, including Multilayer Perceptron, Support Vector Machines, Deep Belief Networks, Long Short-Term Memory Model, Convolutional Neural Network and many other methods in literatures. The comparison results show that ensemble ResCNN achieved the start-of-the-art results, and outperform almost all of them.

References

[1]	M. Ma, C. Sun and X. F. Chen, Discriminative deep belief networks with ant colony optimization for health status assessment of machine, IEEE T. Instrum. Meas., 66 (2017), 3115–3125.
[2]	X. S. Si, W. B. Wang, C. H. Hu and D. H. Zhou, Remaining useful life estimation-a review on the statistical data driven approaches. Eur. J. Oper. Res., 213 (2011), 1–14.
[3]	Y. G. Lei, N. P. Li, L. Guo, N. Li, T. Yan and J. Lin, Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mech. Syst. Signal. Pr., 104 (2018), 799–834.
[4]	K. Javed, R. Gouriveau and N. Zerhouni, A new multivariate approach for prognostics based on extreme learning machine and fuzzy clustering. IEEE Trans. Cybern., 45 (2015), 2626–2639.
[5]	J. B. Ali, B. Chebel-Morello, L. Saidi, S. Malinowski and F. Fnaiech, Accurate bearing remaining useful life prediction based on Weibull distribution and artificial neural network. Mech. Syst. Signal. Pr., 56 (2015), 150–172.
[6]	Y. G. Lei, N. P. Li and J. Lin, A new method based on stochastic process models for machine remaining useful life prediction. IEEE T. Instrum. Meas., 65 (2016), 2671–2684.
[7]	X. Y. Li, C. Lu, L. Gao, S. Q. Xiao and L. Wen, An Effective Multi-Objective Algorithm for Energy Efficient Scheduling in a Real-Life Welding Shop. IEEE T. Ind. Inform., 14, 12(2018), 5400–5409.
[8]	X. Y. Li, L. Gao, Q. Pan, L. Wan and K. M. Chao, An effective hybrid genetic algorithm and variable neighborhood search for integrated process planning and scheduling in a packaging machine workshop. IEEE Trans. Syst., (2018), doi: 10.1109/TSMC.2018.2881686.
[9]	Y. Zhou, W. C. Yi, L. Gao and X. Y. Li, Adaptive differential evolution with sorting crossover rate for continuous optimization problems. IEEE Trans. Cybern., 47 (2017), 2742-2753.
[10]	N. Daroogheh, A. Baniamerian, N. Meskin N and K. Khorasani, Prognosis and health monitoring of nonlinear systems using a hybrid scheme through integration of PFs and neural networks. IEEE Trans. Syst., 47 (2017), 1990–2004.
[11]	R. Q. Huang, L. F. Xi, X. L. Li, C. R. Liu, H. Qiu and J. Lee, Residual life predictions for ball bearings based on self-organizing map and back propagation neural network methods. Mech. Syst. Signal Pr., 21 (2017), 193–207.
[12]	R. Khelif, B. Chebel-Morello, S. Malinowski, E. Laajili, F. Fnaiech and N. Zerhouni, Direct remaining useful life estimation based on support vector regression. IEEE T. Ind. Electron., 64 (2017), 2276–2285.
[13]	C. Ordóñez, F. S. Lasheras, J. Roca-Pardiñas and F. J. de Cos Juez, A hybrid ARIMA-SVM model for the study of the remaining useful life of aircraft engines. J. Comput. Appl. Math., 346 (2019), 184–191.
[14]	H. Z. Huang, H. K. Wang, Y. F. Li, L. Zhang and Z. Liu, Support vector machine based estimation of remaining useful life: Current research status and future trends. J. Mech. Sci. Technol., 29 (2015), 151–163.
[15]	J. Wu, Y. H. Su, Y. W. Cheng, X. Y. Shao, C. Deng and C. Liu, Multi-sensor information fusion for remaining useful life prediction of machining tools by adaptive network based fuzzy inference system. Appl. Soft. Comput., 68, (2018), 13–23.
[16]	C. Chen, B. Zhang, G. Vachtsevanos and M. Orchard, Machine condition prediction based on adaptive neuro-fuzzy and high-order particle filtering. IEEE T. Ind. Electron., 58 (2011), 4353–4364.
[17]	V. Mathew, T. Toby, V. Singh, B. M. Rao and M. G. Kumar, Prediction of Remaining Useful Lifetime (RUL) of turbofan engine using machine learning. 2017 IEEE International Conference on Circuits and Systems (ICCS), 306–311.
[18]	J. L. Wang, J. Zhang and X. X. Wang, A data driven cycle time prediction with feature selection in a semiconductor wafer fabrication system. IEEE T. Semiconduct. M., 31 (2018), 173–182.
[19]	R. Zhao, R. Q. Yan, Z. H. Chen, K. Z. Mao, P. Wang and R. X. Gao, Deep learning and its applications to machine health monitoring. Mech. Syst. Signal. Pr., 115 (2019), 213–237.
[20]	J. Deutsch and D. He, Using deep learning-based approach to predict remaining useful life of rotating components. IEEE Trans. Syst., 48 (2018), 11–20.
[21]	J. Deutsch, M. He and D. He, Remaining useful life prediction of hybrid ceramic bearings using an integrated deep learning and particle filter approach. Appl. Sci., 7 (2017), 649.
[22]	C. Zhang, P. Lim, A. K. Qin and K. C. Tan, Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE Trans. Neural. Netw. Learn. Syst., 28 (2018), 2306–2318.
[23]	L. Wen, L. Gao and X. Y. Li, A new deep transfer learning based on sparse auto-encoder for fault diagnosis. IEEE Trans. Syst., 49 (2019), 136–144.
[24]	H. H. Yan, J. F. Wan, C. H. Zhang, S. L. Tang, Q. S. Hua and Z. R. Wang, Industrial big data analytics for prediction of remaining useful life based on deep learning. IEEE Access, 6 (2018), 17190–17197.
[25]	Y. Y. Zhang, X. Y. Li, L. Gao, L. H. Wang and L. Wen, Imbalanced data fault diagnosis of rotating machinery using synthetic oversampling and feature learning. J. Manuf. Syst., 48 (2018), 34–50.
[26]	F. O. Heimes, Recurrent neural networks for remaining useful life estimation. International Conference on Prognostics and Health Management (PHM 2008), 1–6.
[27]	Y. T. Wu, M. Yuan, S. P. Dong, L. Lin and Y. Q. Liu, Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 275 (2018), 167–179.
[28]	J. L. Wang, J. Zhang and X. X. Wang, Bilateral LSTM: A two-dimensional long short-term memory model with multiply memory units for short-term cycle time forecasting in re-entrant manufacturing systems. IEEE T. Ind. Inform., 14 (2018), 748–758.
[29]	W. N. Lu, Y. P. Li, Y. Cheng, D. S. Meng, B. Liang and P. Zhou, Early fault detection approach with deep architectures. IEEE T. Instrum. Meas., 67 (2018), 1679–1689.
[30]	L. Wen, X. Y. Li and L. Gao, A new convolutional neural network based data-driven fault diagnosis method. IEEE T. Ind. Electron., 65 (2018), 5990–5998.
[31]	G. S. Babu, P. L. Zhao and X. L. Li, Deep convolutional neural network based regression approach for estimation of remaining useful life. Int. Conf. Database Syst. Adv. Appl., (2016), 214–228.
[32]	X. Li, Q. Ding and J. Q. Sun, Remaining useful life estimation in prognostics using deep convolution neural networks. Reliab. Eng. Syst. Safe., 172 (2018), 1–11.
[33]	L. Ren, Y. Q. Sun, H. Wang and L. Zhang, Prediction of bearing remaining useful life with deep convolution neural network. IEEE Access, 6 (2018), 13041–13049.
[34]	L. Guo, Y. G. Lei, N. P. Li, T. Yan and N. B. Li, Machinery health indicator construction based on convolutional neural networks considering trend burr. Neurocomputing, 292 (2018), 142–150.
[35]	A. Z. Hinchi and M. Tkiouat, Rolling element bearing remaining useful life estimation based on a convolutional long-short-term memory network. Procedia. Comput. Sci., 127 (2018), 123–132.
[36]	K. M. He, X. Y. Zhang, S. Q. Ren SQ and J. Sun, Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. Proceedings of the IEEE International Conference on Computer Vision, (2015), 1026–1034.
[37]	K. M. He, X. Y. Zhang, S. Q. Ren, Ren SQ, J. Sun, Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770–778.
[38]	T. W. Rauber, F. Assis Boldt and F. M. Varejão, Heterogeneous feature models and feature selection applied to bearing fault diagnosis. IEEE T. Ind. Electron., 62 (2015), 637–646.
[39]	P. Lim, C. K. Goh and K. C. Tan, A time window neural network based framework for Remaining Useful Life estimation. 2016 International Joint Conference on Neural Networks (IJCNN), 1746–1753.
[40]	Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time series, In: The handbook of brain theory and neural networks, MIT Press Cambridge, MA, USA, 1995.
[41]	Y. Bengio, A. Courville and P. Vincent, Representation learning: A review and new perspectives. IEEE T. Pattern. Anal., 35 (2013), 1798–1828.
[42]	M. Xiao, L. Wen, X. Li and L. Gao, Modeling of the feed-motor transient current in end milling by using varying-coefficient model. Math. Probl. Eng., 2015.
[43]	T. Han, D. Jiang, Q. Zhao Q, L. Wang and K. Yin, Comparison of random forest, artificial neural networks and support vector machine for intelligent diagnosis of rotating machinery. Trans. I. Meas. Control., (2017), 1–13.
[44]	PHM08 Challenge Data Set, NASA Data Repository, 2018. Available from: https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/#turbofan.
[45]	S. Zheng, K. Ristovski, A. Farahat A and C. Gupta, Long short-term memory network for remaining useful life estimation. 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), 88–95.
[46]	S. K. Singh, S. Kumar, J. P. Dwivedi, A novel soft computing method for engine RUL prediction. Multimed. Tools Appl., (2017), 1–23.

Reader Comments

Your name:*

Email:*
© 2019 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)