Reliable prediction of traffic accidents is crucial for the identification of potential hazards in advance, formulation of effective preventative measures, and reduction of accident incidence. Existing neural network-based models generally suffer from a limited field of perception and poor long-term dependency capturing abilities, which severely restrict their performance. To address the inherent shortcomings of current traffic prediction models, we propose the Traffic Transformer for multidimensional, multi-step traffic accident prediction. Initially, raw datasets chronicling sporadic traffic accidents are transformed into multivariate, regularly sampled sequences that are amenable to sequential modeling through a temporal discretization process. Subsequently, Traffic Transformer captures and learns the hidden relationships between any elements of the input sequence, constructing accurate prediction for multiple forthcoming intervals of traffic accidents. Our proposed Traffic Transformer employs the sophisticated multi-head attention mechanism in lieu of the widely used recurrent architecture. This significant shift enhances the model's ability to capture long-range dependencies within time series data. Moreover, it facilitates a more flexible and comprehensive learning of diverse hidden patterns within the sequences. It also offers the versatility of convenient extension and transference to other diverse time series forecasting tasks, demonstrating robust potential for further development in this field. Extensive comparative experiments conducted on a real-world dataset from Qatar demonstrate that our proposed Traffic Transformer model significantly outperforms existing mainstream time series forecasting models across all evaluation metrics and forecast horizons. Notably, its Mean Absolute Percentage Error reaches a minimal value of only 4.43%, which is substantially lower than the error rates observed in other models. This remarkable performance underscores the Traffic Transformer's state-of-the-art level of in predictive accuracy.
Citation: Mansoor G. Al-Thani, Ziyu Sheng, Yuting Cao, Yin Yang. Traffic Transformer: Transformer-based framework for temporal traffic accident prediction[J]. AIMS Mathematics, 2024, 9(5): 12610-12629. doi: 10.3934/math.2024617
Reliable prediction of traffic accidents is crucial for the identification of potential hazards in advance, formulation of effective preventative measures, and reduction of accident incidence. Existing neural network-based models generally suffer from a limited field of perception and poor long-term dependency capturing abilities, which severely restrict their performance. To address the inherent shortcomings of current traffic prediction models, we propose the Traffic Transformer for multidimensional, multi-step traffic accident prediction. Initially, raw datasets chronicling sporadic traffic accidents are transformed into multivariate, regularly sampled sequences that are amenable to sequential modeling through a temporal discretization process. Subsequently, Traffic Transformer captures and learns the hidden relationships between any elements of the input sequence, constructing accurate prediction for multiple forthcoming intervals of traffic accidents. Our proposed Traffic Transformer employs the sophisticated multi-head attention mechanism in lieu of the widely used recurrent architecture. This significant shift enhances the model's ability to capture long-range dependencies within time series data. Moreover, it facilitates a more flexible and comprehensive learning of diverse hidden patterns within the sequences. It also offers the versatility of convenient extension and transference to other diverse time series forecasting tasks, demonstrating robust potential for further development in this field. Extensive comparative experiments conducted on a real-world dataset from Qatar demonstrate that our proposed Traffic Transformer model significantly outperforms existing mainstream time series forecasting models across all evaluation metrics and forecast horizons. Notably, its Mean Absolute Percentage Error reaches a minimal value of only 4.43%, which is substantially lower than the error rates observed in other models. This remarkable performance underscores the Traffic Transformer's state-of-the-art level of in predictive accuracy.
| [1] |
S. Soehodho, Public transportation development and traffic accident prevention in Indonesia, IATSS Res., 40 (2017), 76–80. https://doi.org/10.1016/j.iatssr.2016.05.001 doi: 10.1016/j.iatssr.2016.05.001
|
| [2] |
H. R. Al-Masaeid, A. A. Al-Mashakbeh, A. M. Qudah, Economic costs of traffic accidents in Jordan, Accident Anal. Prev., 31 (1999), 347–357. https://doi.org/10.1016/S0001-4575(98)00068-2 doi: 10.1016/S0001-4575(98)00068-2
|
| [3] | T. Anjuman, S. Hasanat-E-Rabbi, C. K. A. Siddiqui, M. M. Hoque, Road traffic accident: A leading cause of the global burden of public health injuries and fatalities, In: Proceedings of the international conference on mechanical engineering 2007, Bangladesh, 2007. |
| [4] |
A. A. Mohammed, K. Ambak, A. M. Mosa, D. Syamsunur, A review of traffic accidents and related practices worldwide, Open Transport. J., 13 (2019), 65–83. https://doi.org/10.2174/1874447801913010065 doi: 10.2174/1874447801913010065
|
| [5] |
R. Sakhapov, R. Nikolaeva, Traffic safety system management, Transport. Res. Procedia, 36 (2018), 676–681. https://doi.org/10.1016/j.trpro.2018.12.126 doi: 10.1016/j.trpro.2018.12.126
|
| [6] |
K. N. Qureshi, A. H. Abdullah, A survey on intelligent transportation systems, Middle East J. Sci. Res., 15 (2013), 629–642. https://doi.org/10.5829/idosi.mejsr.2013.15.5.11215 doi: 10.5829/idosi.mejsr.2013.15.5.11215
|
| [7] |
B. Lim, S. Zohren, Time-series forecasting with deep learning: A survey, Phil. Trans. R. Soc. A., 379 (2021), 20200209. https://doi.org/10.1098/rsta.2020.0209 doi: 10.1098/rsta.2020.0209
|
| [8] | A. Csikós, Z. J. Viharos, K. B. Kis, T. Tettamanti, I. Varga, Traffic speed prediction method for urban networks–An ANN approach, In: 2015 International conference on models and technologies for intelligent transportation systems (MT-ITS), 2015,102–108. https://doi.org/10.1109/MTITS.2015.7223243 |
| [9] |
M. Y. Çodur, A. Tortum, An artificial neural network model for highway accident prediction: A case study of Erzurum, Turkey, Promet, 27 (2015), 217–225. https://doi.org/10.7307/ptt.v27i3.1551 doi: 10.7307/ptt.v27i3.1551
|
| [10] |
S. Alkheder, M. Taamneh, S. Taamneh, Severity prediction of traffic accident using an artificial neural network, J. Forecast., 36 (2017), 100–108. https://doi.org/10.1002/for.2425 doi: 10.1002/for.2425
|
| [11] |
Z. Sheng, H. Wang, G. Chen, B. Zhou, J. Sun, Convolutional residual network to short-term load forecasting, Appl. Intell., 51 (2021), 2485–2499. https://doi.org/10.1007/s10489-020-01932-9 doi: 10.1007/s10489-020-01932-9
|
| [12] |
S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., 9 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 doi: 10.1162/neco.1997.9.8.1735
|
| [13] |
M. Zheng, T. Li, R. Zhu, J. Chen, Z. Ma, M. Tang, et al., Traffic accident's severity prediction: A deep-learning approach-based CNN network, IEEE Access, 7 (2019), 39897–39910. https://doi.org/10.1109/ACCESS.2019.2903319 doi: 10.1109/ACCESS.2019.2903319
|
| [14] |
D. Yang, S. Li, Z. Peng, P. Wang, J. Wang, H. Yang, MF-CNN: Traffic flow prediction using convolutional neural network and multi-features fusion, IEICE Trans. Inf. Syst., 102 (2019), 1526–1536. https://doi.org/10.1587/transinf.2018EDP7330 doi: 10.1587/transinf.2018EDP7330
|
| [15] |
Z. Zhang, W. Yang, S. Wushour, Traffic accident prediction based on LSTM-GBRT model, J. Control Sci. Eng., 2020 (2020), 4206919. https://doi.org/10.1155/2020/4206919 doi: 10.1155/2020/4206919
|
| [16] | W. Liyong, P. Vateekul, Improve traffic prediction using accident embedding on ensemble deep neural networks, In: 2019 11th International conference on knowledge and smart technology (KST), 2019, 11–16. https://doi.org/10.1109/KST.2019.8687542 |
| [17] |
S. Uğuz, E. Büyükgökoğlan, A hybrid CNN-LSTM model for traffic accident frequency forecasting during the tourist season, Teh. Vjesn., 29 (2022), 2083–2089. https://doi.org/10.17559/TV-20220225141756 doi: 10.17559/TV-20220225141756
|
| [18] |
X. B. Jin, Z. Y. Wang, W. T. Gong, J. L. Kong, Y. T. Bai, T. L. Su, et al., Variational bayesian network with information interpretability filtering for air quality forecasting, Mathematics, 11 (2023), 837. https://doi.org/10.3390/math11040837 doi: 10.3390/math11040837
|
| [19] |
Z. Shi, Y. Bai, X. Jin, X. Wang, T. Su, J. Kong, Parallel deep prediction with covariance intersection fusion on non-stationary time series, Knowl. Based Syst., 211 (2021), 106523. https://doi.org/10.1016/j.knosys.2020.106523 doi: 10.1016/j.knosys.2020.106523
|
| [20] |
X. B. Jin, Z. Y. Wang, J. L. Kong, Y. T. Bai, T. L. Su, H. J. Ma, et al., Deep spatio-temporal graph network with self-optimization for air quality prediction, Entropy, 25 (2023), 247. https://doi.org/10.3390/e25020247 doi: 10.3390/e25020247
|
| [21] |
W. Jiang, J. Luo, Graph neural network for traffic forecasting: A survey, Expert Syst. Appl., 207 (2022), 117921. https://doi.org/10.1016/j.eswa.2022.117921 doi: 10.1016/j.eswa.2022.117921
|
| [22] | A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, In: Advances in neural information processing systems, 30 (2017). |
| [23] | I. Sutskever, O. Vinyals, Q. V. Le, Sequence to sequence learning with neural networks, In: Advances in neural information processing systems, 27 (2014). https://doi.org/10.48550/arXiv.1409.3215 |
| [24] |
P. M. Nadkarni, L. Ohno-Machado, W. W. Chapman, Natural language processing: An introduction, J. Amer. Med. Inform. Assoc., 18 (2011), 544–551. https://doi.org/10.1136/amiajnl-2011-000464 doi: 10.1136/amiajnl-2011-000464
|
| [25] |
A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis, Deep learning for computer vision: A brief review, Comput. Intell. Neurosci., 2018 (2018), 7068349. https://doi.org/10.1155/2018/7068349 doi: 10.1155/2018/7068349
|
| [26] | Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, et al., Transformers in time series: A survey, arXiv: 2202.07125, 2022. https://doi.org/10.48550/arXiv.2202.07125 |
| [27] | A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv: 2010.11929, 2020. https://doi.org/10.48550/arXiv.2010.11929 |
| [28] |
H. Yin, Z. Guo, X. Zhang, J. Chen, Y. Zhang, RR-Former: Rainfall-runoff modeling based on Transformer, J. Hydrology, 609 (2022), 127781. https://doi.org/10.1016/j.jhydrol.2022.127781 doi: 10.1016/j.jhydrol.2022.127781
|
| [29] |
G. Zheng, W. K. Chai, J. Zhang, V. Katos, VDGCNeT: A novel network-wide virtual dynamic graph convolution neural network and Transformer-based traffic prediction model, Knowl. Based Syst., 275 (2023), 110676. https://doi.org/10.1016/j.knosys.2023.110676 doi: 10.1016/j.knosys.2023.110676
|
| [30] |
Z. Sheng, S. Wen, Z. K. Feng, J. Gong, K. Shi, Z. Guo, et al., A survey on data-driven runoff forecasting models based on neural networks, IEEE Trans. Emerg. Top. Comput. Intell., 7 (2023), 1083–1097. https://doi.org/10.1109/TETCI.2023.3259434 doi: 10.1109/TETCI.2023.3259434
|
| [31] |
Z. Li, F. Liu, W. Yang, S. Peng, J. Zhou, A survey of convolutional neural networks: Analysis, applications, and prospects, IEEE Trans. Neural Netw. Learn. Syst., 33 (2021), 6999–7019. https://doi.org/10.1109/TNNLS.2021.3084827 doi: 10.1109/TNNLS.2021.3084827
|
| [32] |
Y. Yu, X. Si, C. Hu, J. Zhang, A review of recurrent neural networks: LSTM cells and network architectures, Neural Comput., 31 (2019), 1235–1270. https://doi.org/10.1162/neco_a_01199 doi: 10.1162/neco_a_01199
|
| [33] | K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), 2016,770–778. https://doi.org/10.1109/CVPR.2016.90 |
| [34] | J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization, arXiv: 1607.06450, 2016. https://doi.org/10.48550/arXiv.1607.06450 |
| [35] | A. F. Agarap, Deep learning using rectified linear units (relu), arXiv: 1803.08375, 2018. https://doi.org/10.48550/arXiv.1803.08375 |
| [36] | D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv: 1412.6980, 2014. https://doi.org/10.48550/arXiv.1412.6980 |
Figures(5) / Tables(3)
Mansoor G. Al-Thani, Ziyu Sheng, Yuting Cao, Yin Yang. Traffic Transformer: Transformer-based framework for temporal traffic accident prediction[J]. AIMS Mathematics, 2024, 9(5): 12610-12629. doi: 10.3934/math.2024617
DownLoad: