Traffic flow prediction with a multi-dimensional feature input: A new method based on attention mechanisms

Shaohu Zhang; Jianxiao Ma; Boshuo Geng; Hanbin Wang; Shaohu Zhang; Jianxiao Ma; Boshuo Geng; Hanbin Wang

doi:10.3934/era.2024048

Electronic Research Archive

2024, Volume 32, Issue 2: 979-1002. doi: 10.3934/era.2024048

Previous Article Next Article

Research article Special Issues

Traffic flow prediction with a multi-dimensional feature input: A new method based on attention mechanisms

College of Automobile and Traffic Engineering, Nanjing Forestry University, Nanjing 210037, China

Received: 19 October 2023 Revised: 06 December 2023 Accepted: 04 January 2024 Published: 19 January 2024

Accurately predicting traffic flow is an essential component of intelligent transportation systems. The advancements in traffic data collection technology have broadened the range of features that affect and represent traffic flow variations. However, solely inputting gathered features into the model without analysis might overlook valuable information, hindering the improvement of predictive performance. Furthermore, intricate dynamic relationships among various feature inputs could constrain the model's potential for further enhancement in predictive accuracy. Consequently, extracting pertinent features from datasets and modeling their mutual influence is critical in attaining heightened precision in traffic flow predictions. First, we perform effective feature extraction by considering the temporal dimension and inherent operating rules of traffic flow, culminating in Multivariate Time Series (MTS) data used as input for the model. Then, an attention mechanism is proposed based on the MTS input data. This mechanism assists the model in selecting pertinent time series for multivariate forecasting, mitigating inter-feature influence, and achieving accurate predictions through the concentration on crucial information. Finally, empirical findings from real highway datasets illustrate the enhancement of predictive accuracy attributed to the proposed features within the model. In contrast to conventional machine learning or attention-based deep learning models, the proposed attention mechanism in this study demonstrates superior accuracy and stability in MTS-based traffic flow prediction tasks.
- intelligent transportation system,
- traffic flow modeling,
- time series analysis,
- deep learning,
- attention mechanism
Citation: Shaohu Zhang, Jianxiao Ma, Boshuo Geng, Hanbin Wang. Traffic flow prediction with a multi-dimensional feature input: A new method based on attention mechanisms[J]. Electronic Research Archive, 2024, 32(2): 979-1002. doi: 10.3934/era.2024048

Related Papers:

Abstract

Accurately predicting traffic flow is an essential component of intelligent transportation systems. The advancements in traffic data collection technology have broadened the range of features that affect and represent traffic flow variations. However, solely inputting gathered features into the model without analysis might overlook valuable information, hindering the improvement of predictive performance. Furthermore, intricate dynamic relationships among various feature inputs could constrain the model's potential for further enhancement in predictive accuracy. Consequently, extracting pertinent features from datasets and modeling their mutual influence is critical in attaining heightened precision in traffic flow predictions. First, we perform effective feature extraction by considering the temporal dimension and inherent operating rules of traffic flow, culminating in Multivariate Time Series (MTS) data used as input for the model. Then, an attention mechanism is proposed based on the MTS input data. This mechanism assists the model in selecting pertinent time series for multivariate forecasting, mitigating inter-feature influence, and achieving accurate predictions through the concentration on crucial information. Finally, empirical findings from real highway datasets illustrate the enhancement of predictive accuracy attributed to the proposed features within the model. In contrast to conventional machine learning or attention-based deep learning models, the proposed attention mechanism in this study demonstrates superior accuracy and stability in MTS-based traffic flow prediction tasks.

References

[1]	Z. Ge, Y. Li, C. Liang, Y. Song, T. Zhou, J. Qin, Acsnet: adaptive cross-scale network with feature maps refusion for vehicle density detection, in 2021 IEEE International Conference on Multimedia and Expo (ICME), IEEE, (2021), 1–6. https://doi.org/10.1109/ICME51207.2021.9428454
[2]	H. Lu, Z. Ge, Y. Song, D. Jiang, T. Zhou, J. Qin, A temporal-aware LSTM enhanced by loss-switch mechanism for traffic flow forecasting, Neurocomputing, 427 (2021), 169–178. https://doi.org/10.1016/j.neucom.2020.11.026 doi: 10.1016/j.neucom.2020.11.026
[3]	B. S. Chen, S. C. Peng, K. C. Wang, Traffic modeling, prediction, and congestion control for high-speed networks: A fuzzy AR approach, IEEE Trans. Fuzzy Syst., 8 (2000), 491–508. https://doi.org/10.1109/91.873574 doi: 10.1109/91.873574
[4]	B. M. Williams, L. A. Hoel, Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results, J. Transp. Eng., 129 (2003), 664–672.
[5]	L. Moreira-Matias, J. Gama, M. Ferreira, J. Mendes-Moreira, L. Damas, Predicting taxi-passenger demand using streaming data, IEEE Trans. Intell. Transp. Syst., 14 (2013), 1393–1402. https://doi.org/10.1109/TITS.2013.2262376 doi: 10.1109/TITS.2013.2262376
[6]	F. Wu, H. Wang, Z. Li, Interpreting traffic dynamics using ubiquitous urban data, in Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2016. https://doi.org/10.1145/2996913.2996962
[7]	N. P. Trinh, A. K. N. Tran, T. H. Do, Traffic flow forecasting using multivariate time-series deep learning and distributed computing, in 2022 RIVF International Conference on Computing and Communication Technologies (RIVF), IEEE, (2022), 665–670. https://doi.org/10.1109/RIVF55975.2022.10013796
[8]	B. Geng, J. Ma, S. Zhang, Ensemble deep learning-based lane-changing behavior prediction of manually driven vehicles in mixed traffic environments, Electron. Res. Arch., 31 (2023), 6216–6235. https://doi.org/10.3934/era.2023315 doi: 10.3934/era.2023315
[9]	F. Aljuaydi, B. Wiwatanapataphee, Y. H. Wu, Multivariate machine learning-based prediction models of freeway traffic flow under non-recurrent events, Alexandria Eng. J., 65 (2023), 151–162.
[10]	C. Chen, Z. Liu, S. Wan, J. Luan, Q. Pei, Traffic flow prediction based on deep learning in internet of vehicles, IEEE Trans. Intell. Transp. Syst., 22 (2020), 3776–3789. https://doi.org/10.1109/TITS.2020.3025856 doi: 10.1109/TITS.2020.3025856
[11]	J. An, L. Fu, M. Hu, W. Chen, J. Zhan, A novel fuzzy-based convolutional neural network method to traffic flow prediction with uncertain traffic accident information, IEEE Access, 7 (2019), 20708–20722. https://doi.org/10.1109/ACCESS.2019.2896913 doi: 10.1109/ACCESS.2019.2896913
[12]	Y. Li, S. Chai, Z. Ma, G. Wang, A hybrid deep learning framework for long-term traffic flow prediction, IEEE Access, 9 (2021), 11264–11271. https://doi.org/10.1109/ACCESS.2021.3050836 doi: 10.1109/ACCESS.2021.3050836
[13]	X. Wang, Y. Wang, J. Peng, Z. Zhang, X. Tang, A hybrid framework for multivariate long-sequence time series forecasting, Appl. Intell., 53 (2023), 13549–13568. https://doi.org/10.1007/s10489-022-04110-1 doi: 10.1007/s10489-022-04110-1
[14]	Y. Zhang, Y. Yang, W. Zhou, H. Wang, X. Ouyang, Multi-city traffic flow forecasting via multi-task learning, Appl. Intell., 2021 (2021), 1–19. https://doi.org/10.1007/s10489-020-02074-8 doi: 10.1007/s10489-020-02074-8
[15]	M. Méndez, M. G. Merayo, M. Núñez, Long-term traffic flow forecasting using a hybrid CNN-BiLSTM model, Eng. Appl. Artif. Intell., 121 (2023), 106041. https://doi.org/10.1016/j.engappai.2023.106041 doi: 10.1016/j.engappai.2023.106041
[16]	Q. Du, F. Yin, Z. Li, Base station traffic prediction using XGBoost‐LSTM with feature enhancement, IET Networks, 9 (2020), 29–37. https://doi.org/10.1049/iet-net.2019.0103 doi: 10.1049/iet-net.2019.0103
[17]	S. Wang, M. Zhang, H. Miao, Z. Peng, P. S. Yu, Multivariate correlation-aware spatio-temporal graph convolutional networks for multi-scale traffic prediction, ACM Trans. Intell. Syst. Technol., 13 (2022), 1–22. https://doi.org/10.1145/3469087 doi: 10.1145/3469087
[18]	Y. Li, K. Li, C. Chen, X. Zhou, Z. Zeng, K. Li, Modeling temporal patterns with dilated convolutions for time-series forecasting, ACM Trans. Knowl. Discovery Data, 16 (2021), 1–22. https://doi.org/10.1145/3453724 doi: 10.1145/3453724
[19]	Q. Zhao, G. Yang, K. Zhao, J. Yin, W. Rao, L. Chen, Multivariate time-series forecasting model: Predictability analysis and empirical study, IEEE Trans. Big Data, 2023. https://doi.org/10.1109/TBDATA.2023.3288693 doi: 10.1109/TBDATA.2023.3288693
[20]	L. N. Do, H. L. Vu, B. Q. Vo, Z. Liu, D. Phung, An effective spatial-temporal attention based neural network for traffic flow prediction, Transp. Res. Part C Emerging Technol., 108 (2019), 12–28. https://doi.org/10.1016/j.trc.2019.09.008 doi: 10.1016/j.trc.2019.09.008
[21]	H. Zheng, F. Lin, X. Feng, Y. Chen, A hybrid deep learning model with attention-based conv-LSTM networks for short-term traffic flow prediction, IEEE Trans. Intell. Transp. Syst., 22 (2020), 6910–6920. https://doi.org/10.1109/TITS.2020.2997352 doi: 10.1109/TITS.2020.2997352
[22]	J. Wu, J. Fu, H. Ji, L. Liu, Graph convolutional dynamic recurrent network with attention for traffic forecasting, Appl. Intell., 2023 (2023), 1–15. https://doi.org/10.1007/s10489-023-04621-5 doi: 10.1007/s10489-023-04621-5
[23]	D. Qin, Z. Peng, L. Wu, Deep attention fuzzy cognitive maps for interpretable multivariate time series prediction, Knowl.-Based Syst., (2023), 110700. https://doi.org/10.1016/j.knosys.2023.110700 doi: 10.1016/j.knosys.2023.110700
[24]	W. Fang, W. Zhuo, J. Yan, Y. Song, D. Jiang, T. Zhou, Attention meets long short-term memory: A deep learning network for traffic flow forecasting, Physica A, 587 (2022), 126485. https://doi.org/10.1016/j.physa.2021.126485 doi: 10.1016/j.physa.2021.126485
[25]	R. Wan, C. Tian, W. Zhang, W. Deng, F. Yang, A multivariate temporal convolutional attention network for time-series forecasting, Electronics, 11 (2022), 1516. https://doi.org/10.3390/electronics11101516 doi: 10.3390/electronics11101516
[26]	S. Shun-Yao, S. Fan-Keng, L. Hung-yi, Temporal pattern attention for multivariate time series forecasting, Mach. Learn., 108 (2019). https://doi.org/10.1007/s10994-019-05815-0 doi: 10.1007/s10994-019-05815-0
[27]	X. Geng, X. He, L. Xu, J. Yu, Graph correlated attention recurrent neural network for multivariate time series forecasting, Inf. Sci., 606 (2022), 126–142. https://doi.org/10.1016/j.ins.2022.04.045 doi: 10.1016/j.ins.2022.04.045
[28]	D. Cao, Y. Wang, J. Duan, C. Zhang, X. Zhu, C. Huang, et al., Spectral temporal graph neural network for multivariate time-series forecasting, Adv. Neural Inf. Process. Syst., 33 (2020), 17766–17778.
[29]	X. Liu, J. Xu, M. Li, L. Wei, H. Ru, General-logistic-based speed-density relationship model incorporating the effect of heavy vehicles, Math. Probl. Eng., 2019 (2019). https://doi.org/10.1155/2019/6039846 doi: 10.1155/2019/6039846
[30]	S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., 9 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 doi: 10.1162/neco.1997.9.8.1735
[31]	R. Pascanu, T. Mikolov, Y. Bengio, On the difficulty of training recurrent neural networks, in International Conference on Machine Learning, PMLR, (2013), 1310–1318.
[32]	A. Graves, Generating sequences with recurrent neural networks, preprint, arXiv: 13080850. https://doi.org/10.48550/arXiv.1308.0850
[33]	L. Cai, M. Lei, S. Zhang, Y. Yu, T. Zhou, J. Qin, A noise-immune LSTM network for short-term traffic flow forecasting, Chaos, 30 (2020). https://doi.org/10.1063/1.5120502 doi: 10.1063/1.5120502
[34]	D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, preprint, arXiv: 14090473. https://doi.org/10.48550/arXiv.1409.0473
[35]	M. T. Luong, H. Pham, C. D. Manning, Effective approaches to attention-based neural machine translation, preprint, arXiv: 150804025. https://doi.org/10.48550/arXiv.1508.04025
[36]	Y. Lv, Y. Duan, W. Kang, Z. Li, F. Y. Wang, Traffic flow prediction with big data: A deep learning approach, IEEE Trans. Intell. Transp. Syst., 16 (2014), 865–873. https://doi.org/10.1109/TITS.2014.2345663 doi: 10.1109/TITS.2014.2345663
[37]	W. Chai, Y. Zheng, L. Tian, J. Qin, T. Zhou, GA-KELM: Genetic-algorithm-improved kernel extreme learning machine for traffic flow forecasting, Mathematics, 11 (2023), 3574. https://doi.org/10.3390/math11163574 doi: 10.3390/math11163574
[38]	M. A. Dulebenets, A Diffused Memetic Optimizer for reactive berth allocation and scheduling at marine container terminals in response to disruptions, Swarm Evol. Comput., 80 (2023), 101334. https://doi.org/10.1016/j.swevo.2023.101334 doi: 10.1016/j.swevo.2023.101334
[39]	M. A. Dulebenets, An Adaptive Polyploid Memetic Algorithm for scheduling trucks at a cross-docking terminal, Inf. Sci., 565 (2021), 390–421. https://doi.org/10.1016/j.ins.2021.02.039 doi: 10.1016/j.ins.2021.02.039
[40]	M. Chen, Y. Tan, SF-FWA, A Self-Adaptive Fast Fireworks Algorithm for effective large-scale optimization, Swarm Evol. Comput., 80 (2023), 101314. https://doi.org/101016/jswevo
[41]	J. Pasha, A. L. Nwodu, A. M. Fathollahi-Fard, G. Tian, Z. Li, H. Wang, et al., Exact and metaheuristic algorithms for the vehicle routing problem with a factory-in-a-box in multi-objective settings, Adv. Eng. Inf., 52 (2022), 101623. https://doi.org/10.1016/j.aei.2022.101623 doi: 10.1016/j.aei.2022.101623

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)