Research and comparison of pavement performance prediction based on neural networks and fusion transformer architecture

Hui Yao; Ke Han; Yanhao Liu; Dawei Wang; Zhanping You; Hui Yao; Ke Han; Yanhao Liu; Dawei Wang; Zhanping You

doi:10.3934/era.2024059

Electronic Research Archive

2024, Volume 32, Issue 2: 1239-1267. doi: 10.3934/era.2024059

Previous Article Next Article

Research article Special Issues

Research and comparison of pavement performance prediction based on neural networks and fusion transformer architecture

1.
Beijing Key Laboratory of Traffic Engineering, College of Metropolitan Transportation, Faculty of Architecture, Civil and Transportation Engineering, Beijing University of Technology, No.100, Pingleyuan, Chaoyang, Beijing 100124, China
2.
School of Transportation Science and Engineering, Harbin Institute of Technology, Harbin 150006, China
3.
Department of Civil and Environmental Engineering, Michigan Technological University, 1400 Townsend Drive, Houghton, MI 49931-1295, United States

Academic Editor: Victor Barranca

Received: 11 November 2023 Revised: 03 January 2024 Accepted: 19 January 2024 Published: 30 January 2024

The decision-making process for pavement maintenance from a scientific perspective is based on accurate predictions of pavement performance. To improve the rationality of pavement performance indicators, comprehensive consideration of various influencing factors is necessary. To this end, four typical pavement performance indicators (i.e., Rutting Depth, International Roughness Index, Longitudinal Cracking, and Alligator Cracking) were predicted using the Long Term Pavement Performance (LTPP) database. Two types of data, i.e., local input variables and global input variables, were selected, and S-ANN and L-ANN models were constructed using a fully connected neural network. A comparative analysis of the predictive outcomes reveals the superior optimization of the L-ANN model. Subsequently, by incorporating structures such as self-attention mechanism, a novel predictive approach based on the Transformer architecture was proposed. The objective is to devise a more accurate predictive methodology for pavement performance indices, with the goal of guiding pavement maintenance and management efforts. Experimental results indicate that, through comparative analysis of three quantitative evaluation metrics (root mean square error, mean absolute error, coefficient of determination), along with visual scatter plots, the predictive model employing the fused Transformer architecture demonstrates higher robustness and accuracy within the domain of pavement performance prediction when compared to the L-ANN model. This outcome substantiates the efficacy and superiority of the model in terms of predictive performance, establishing it as a reliable tool for accurately reflecting the evolution of asphalt pavement performance. Furthermore, it furnishes a theoretical reference for determining optimal preventive maintenance timing for pavements.
- pavement engineering,
- performance prediction,
- transformer,
- neural network,
- self-attention
Citation: Hui Yao, Ke Han, Yanhao Liu, Dawei Wang, Zhanping You. Research and comparison of pavement performance prediction based on neural networks and fusion transformer architecture[J]. Electronic Research Archive, 2024, 32(2): 1239-1267. doi: 10.3934/era.2024059

Related Papers:

Abstract

The decision-making process for pavement maintenance from a scientific perspective is based on accurate predictions of pavement performance. To improve the rationality of pavement performance indicators, comprehensive consideration of various influencing factors is necessary. To this end, four typical pavement performance indicators (i.e., Rutting Depth, International Roughness Index, Longitudinal Cracking, and Alligator Cracking) were predicted using the Long Term Pavement Performance (LTPP) database. Two types of data, i.e., local input variables and global input variables, were selected, and S-ANN and L-ANN models were constructed using a fully connected neural network. A comparative analysis of the predictive outcomes reveals the superior optimization of the L-ANN model. Subsequently, by incorporating structures such as self-attention mechanism, a novel predictive approach based on the Transformer architecture was proposed. The objective is to devise a more accurate predictive methodology for pavement performance indices, with the goal of guiding pavement maintenance and management efforts. Experimental results indicate that, through comparative analysis of three quantitative evaluation metrics (root mean square error, mean absolute error, coefficient of determination), along with visual scatter plots, the predictive model employing the fused Transformer architecture demonstrates higher robustness and accuracy within the domain of pavement performance prediction when compared to the L-ANN model. This outcome substantiates the efficacy and superiority of the model in terms of predictive performance, establishing it as a reliable tool for accurately reflecting the evolution of asphalt pavement performance. Furthermore, it furnishes a theoretical reference for determining optimal preventive maintenance timing for pavements.

References

[1]	H. Yao, Y. Liu, X. Li, Z. You, Y. Feng, W. Lu, A detection method for pavement cracks combining object detection and attention mechanism, IEEE Trans. Intell. Transp. Syst., 23 (2022), 22179–22189. https://doi.org/10.1109/TITS.2022.3177210 doi: 10.1109/TITS.2022.3177210
[2]	H. Yao, Y. Liu, H. Lv, J. Huyan, Z. You, Y. Hou, Encoder-decoder with pyramid region attention for pixel-level pavement crack recognition, Comput.-Aided Civil Infrastruct. Eng., 2023. https://doi.org/10.1111/mice.13128 doi: 10.1111/mice.13128
[3]	M. Zhu, K. Bi, H. Yu, X. Liu, W. Qiao, Research on pavement structure and performance prediction based on long-life test sectio, J. Muni. Tech., 41 (2023), 58–65. https://doi.org/10.19922/j.1009-7767.2023.05.058 doi: 10.19922/j.1009-7767.2023.05.058
[4]	Y. Jin, T. Shen, Y. Tang, L. Li, Pavement performance prediction based on toll date: Case study in Shaanxi Province, China, in 18th COTA International Conference of Transportation Professionals, 13 (2018), 545–554. https://doi.org/10.1061/9780784481523.055 doi: 10.1061/9780784481523.055
[5]	J. Karam, H. Noorvand, Developing a rutting prediction model for HMA pavements using the LTPP database, Int. J. Pavement Res. Technol., 2023. https://doi.org/10.1007/s42947-023-00340-9 doi: 10.1007/s42947-023-00340-9
[6]	Z. Hen, X. Sun, W. Yang, Q. Li, G. Xiao, S. Xu, Performance prediction model and accuracy analysis of asphalt pavement of sixth ring freeway in Beijing, J. Muni. Tech., 38 (2020), 34–36.
[7]	K. A. Abaza, Back-calculation of transition probabilities for Markovian-based pavement performance prediction models, Int. J. Pavement Res. Technol., 17 (2016), 253–264. https://doi.org/10.1080/10298436.2014.993185 doi: 10.1080/10298436.2014.993185
[8]	K. A. Abaza, Simplified Markovian-based pavement management model for sustainable long-term rehabilitation planning, Road Mater. Pavement Des., 24 (2023), 850–865. https://doi.org/10.1080/14680629.2022.2048055 doi: 10.1080/14680629.2022.2048055
[9]	L. Yao, Q. Dong, J. Jiang, F. Ni, Establishment of prediction models of asphalt pavement performance based on a novel data calibration method and neural network, Transp. Res. Rec., 2673 (2019), 66–82. https://doi.org/10.1177/0361198118822501 doi: 10.1177/0361198118822501
[10]	A. A. Ali, U. Heneash, A. Hussein, S. Khan, Application of Artificial neural network technique for prediction of pavement roughness as a performance indicator, J. King Saud Univ. Sci., 2023. https://doi.org/10.1016/j.jksues.2023.01.001 doi: 10.1016/j.jksues.2023.01.001
[11]	Y. Zhu, J. Chen, K. Wang, Y. Liu, Y. Wang, Research on performance prediction of highway asphalt pavement based on Grey–Markov model, Transp. Res. Rec., 2676 (2021), 194–209. https://doi.org/10.1177/03611981211057527 doi: 10.1177/03611981211057527
[12]	Y. Song, YD. Wang, X. Hu, J. Liu, An efficient and explainable ensemble learning model for asphalt pavement condition prediction based on LTPP dataset, IEEE Trans. Intell. Transp. Syst., 23 (2022), 22084–22093. https://doi.org/10.1109/TITS.2022.3164596 doi: 10.1109/TITS.2022.3164596
[13]	G. Liu, F. Niu, Z. Wu, Life-cycle performance prediction for rigid runway pavement using artificial neural network, Int. J. Pavement Res. Technol., 21 (2020), 1806–1814. https://doi.org/10.1080/10298436.2019.1567922 doi: 10.1080/10298436.2019.1567922
[14]	K. Othman, Prediction of the hot asphalt mix properties using deep neural networks, Beni-Suef Univ. J. Basic Appl. Sci., 11 (2022), 40. https://doi.org/10.1186/s43088-022-00221-3 doi: 10.1186/s43088-022-00221-3
[15]	Q. Dong, X. Chen, S. Dong, Classification of pavement climatic regions through unsupervised and supervised machine learnings, J. Infrastruct. Preserv. Resilience, 2 (2021), 5. https://doi.org/10.1186/s43065-021-00020-7 doi: 10.1186/s43065-021-00020-7
[16]	Z. Sun, X. Hao, W. Li, J. Huyan, H. Sun, Asphalt pavement friction coefficient prediction method based on genetic-algorithm-improved neural network (GAI-NN) model, Can. J. Civ. Eng., 49 (2022), 109–120. https://doi.org/10.1139/cjce-2020-0051 doi: 10.1139/cjce-2020-0051
[17]	M. Mers, Z. Yang, Y. A. Hsieh, Y. Tsai, Recurrent neural networks for pavement performance forecasting: review and model performance comparison, Transp. Res. Rec., 2677 (2022), 610–624. https://doi.org/10.1177/03611981221100521 doi: 10.1177/03611981221100521
[18]	M. I. Hossain, L. S. P. Gopisetti, M. S. Miah, Prediction of international roughness index of flexible pavements from climate and traffic data using artificial neural network modeling, Airfield Highw. Pavements 2017, (2017), 256–267. https://doi.org/10.1061/9780784480922.023 doi: 10.1061/9780784480922.023
[19]	B. Rulian, Y. Hakan, S. Salma, M. J. Z. Fanhmi, N. Yacoub, Performance model development for flexible pavements via neural networks, in International Conference on Transportation and Development, 5 (2022), 73–84. https://doi.org/10.1061/9780784484357.007
[20]	J. Liu, F. Liu, C. Zheng, E. Fanijo, L. Wang, Improving asphalt mix design considering international roughness index of asphalt pavement predicted using autoencoders and machine learning, Constr. Build. Mater., 360 (2022), 129439. https://doi.org/10.1016/j.conbuildmat.2022.129439 doi: 10.1016/j.conbuildmat.2022.129439
[21]	M. M. Radwan, M. A. Abo-Hashema, H. P. Faheem, M. D. Hashem, ANN-based fatigue and rutting prediction models versus regression-based models for flexible pavements, in 3rd GeoMEast International Congress and Exhibition on Sustainable Civil Infrastructures, (2020), 117–133. https://doi.org/10.1007/978-3-030-34196-1_9
[22]	M. Mahmood, U. Anuraj, S. Mathavan, M. Rahman, A unified artificial neural network model for asphalt pavement condition prediction, Proc. Inst. Civil Eng.-Transp., 176 (2023), 14–24. https://doi.org/10.1680/jtran.19.00111 doi: 10.1680/jtran.19.00111
[23]	Z. Luo, S. Li, An interpretable prediction model for pavement performance prediction based on XGBoost and SHAP, in Second International Conference on Electronic Information Engineering and Computer Communication (EIECC 2022), 12594 (2022), 187–194. https://doi.org/10.1117/12.2671361
[24]	A. L. Aranha, L. L. B. Bernucci, K. L. Vasconcelos, Effects of different training datasets on machine learning models for pavement performance prediction, Transp. Res. Rec., 2677 (2023), 196–206. https://doi.org/10.1177/03611981231155902 doi: 10.1177/03611981231155902
[25]	O. Kaya, H. Ceylan, S. Kim, D. Waid, B. P. Moore, Statistics and artificial intelligence-based pavement performance and remaining service life prediction models for flexible and composite pavement systems, Transp. Res. Rec., 2674 (2020), 448–460. https://doi.org/10.1177/0361198120915889 doi: 10.1177/0361198120915889
[26]	M. Xiao, R. Luo, Y. Chen, X. Ge, Prediction model of asphalt pavement functional and structural performance using PSO-BPNN algorithm, Constr. Build. Mater., 407 (2023), 133534. https://doi.org/10.1016/j.conbuildmat.2023.133534 doi: 10.1016/j.conbuildmat.2023.133534
[27]	S. Saha, F. Gu, X. Luo, RL. Lytton, Development of an artificial neural network-based k-value prediction model to improve the sensitivity of base layer on rigid pavement performance, Transp. Res. Rec., 2677 (2023), 1290–1308. https://doi.org/ 10.1177/03611981221143114 doi: 10.1177/03611981221143114
[28]	G. Liu, F. Niu, Z. Wu, Life-cycle performance prediction for rigid runway pavement using artificial neural network, Int. J. Pavement Eng., 21 (2020), 1806–1814. https://doi.org/10.1080/10298436.2019.1567922 doi: 10.1080/10298436.2019.1567922
[29]	N. Wu, B. Green, X. Ben, Deep transformer models for time series forecasting: The influenza prevalence case, preprint, arXiv: 2001.08317.
[30]	Q. Zhou, E. Okte, I. L. Al-Qadi, Predicting pavement roughness using deep learning algorithms, Transp. Res. Rec., 2675 (2021), 1062–1072. https://doi.org/10.1177/03611981211023765 doi: 10.1177/03611981211023765
[31]	J. Xin, M. Akiyama, DM. Frangopol, Sustainability-informed management optimization of asphalt pavement considering risk evaluated by multiple performance indicators using deep neural networks, Reliab. Eng. Syst. Saf., 238 (2023), 109488. https://doi.org/10.1016/j.ress.2023.109448 doi: 10.1016/j.ress.2023.109448
[32]	S. Salma, Y. HaKan, B. Rulian, N. Jacob, Evaluating the effect of climate change in pavement performance modeling using artificial neural network approach, in International Conference on Transportation and Development, 5 (2022), 49–60. https://doi.org/10.1061/9780784484357.005
[33]	J. Lucey, A. Fathi, M. Mazari, Predicting pavement roughness as a performance indicator using historical data and artificial intelligence, in International Airfield and Highway Pavements Conference 2019, (2019), 10–18. https://doi.org/10.1061/9780784482476.002
[34]	J. Xin, M. Akiyama, DM. Frangopol, M. Zhang, Multi-objective optimisation of in-service asphalt pavement maintenance schedule considering system reliability estimated via LSTM neural networks, Struct. Infrastruct. Eng., 18 (2022), 1002–1019. https://doi.org/10.1080/15732479.2022.2038641 doi: 10.1080/15732479.2022.2038641
[35]	A. Das, W. Kong, A. L. Leach, R. Sen, R. Yu, Long-term Forecasting with TiDE: Time-series Dense Encoder, preprint, arXiv: 2304.08424.
[36]	S. Bai, W. Yang, M. Zhang, D. Liu, W. Li, L. Zhou, Attention-based BiLSTM model for pavement temperature prediction of asphalt pavement in winter, Atmosphere, 13 (2022), 1542. https://doi.org/10.3390/atmos13091524 doi: 10.3390/atmos13091524
[37]	F. Guo, Y. Qian, Intelligent pavement roughness forecasting based on a long short-term memory model with attention mechanism, Airfield Highw. Pavements, (2021), 128–136. https://doi.org/10.1061/9780784483503.013 doi: 10.1061/9780784483503.013
[38]	X. Tong, Y. Dong, Y. Zhang, Pavement maintenance plan and practice based on pavement performance prediction, J. Muni. Tech., 39 (2021), 30–34. https://doi.org/10.19922/j.1009-7767.2021.07.028 doi: 10.19922/j.1009-7767.2021.07.028
[39]	A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, Adv. Neural Inf. Process. Syst., 30 (2017). https://doi.org/10.48550/arXiv.1706.03762 doi: 10.48550/arXiv.1706.03762

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)