Covariate-aware transformer architectures for long-horizon demand forecasting in Retail analytics

Masad A. Alrasheedi; Asamh Saleh M. Al Luhayb; Nasser Aedh Alreshidi; Masad A. Alrasheedi; Asamh Saleh M. Al Luhayb; Nasser Aedh Alreshidi

doi:10.3934/math.2026885

AIMS Mathematics

2026, Volume 11, Issue 7: 21885-21927. doi: 10.3934/math.2026885

Previous Article Next Article

Research article Special Issues

Covariate-aware transformer architectures for long-horizon demand forecasting in Retail analytics

1.
Department of Management Information Systems, College of Business Administration, Taibah University, Madinah, Saudi Arabia
2.
Department of Mathematics, College of Science, Qassim University, P.O. Box 6644, Buraydah, 51452, Saudi Arabia
3.
Department of Mathematics, College of Science, Northern Border University, Arar 73213, Saudi Arabia

Received: 27 April 2026 Revised: 30 June 2026 Accepted: 13 July 2026 Published: 23 July 2026
MSC : 68T07, 68T09, 62M10, 90B50, 90C90

Demand forecasting from a retail perspective affects various facets of the business, including workforce planning, inventory control, and supply chain management, and informs key business strategies. The long-range demand forecasting techniques evaluate various external influences and deal with time series data that often exhibit irregular patterns and complex seasonality, and are often hierarchical. In this work, we proposed a long-range demand forecasting framework that used covariates and was based on transformer architectures with the following key components: multihead self-attention, direct multi-horizon prediction, positional encodings, and categorical embeddings. This framework can model various temporal patterns and was context-aware within the retail system. Having implemented the framework, we benchmarked it against state-of-the-art demand forecasting models on several retail forecasting datasets, as well as other complex time-series, multi-dimensional forecasting datasets. The model empirically demonstrated a 3.1% improvement on the retail forecasting benchmark (achieving a weighted root mean squared scaled error (WRMSSE) of 0.498) and a 1.7% improvement on the benchmark for complex time series datasets (achieving an symmetric mean absolute percentage error (sMAPE) of 12.47) compared to the leading transformer models. Additionally, we provided empirical evidence of enhanced demand forecasting, improved stability and robustness, reduced training time, and improved model throughput. With all of these model features and enhancements combined, we believe this will make retail demand forecasting significantly more feasible and add considerable operational value, particularly for the planning and strategy aspects of retail management.
- long-horizon forecasting,
- retail demand prediction,
- transformer models,
- time-series analytics,
- hierarchical forecasting,
- deep learning
Citation: Masad A. Alrasheedi, Asamh Saleh M. Al Luhayb, Nasser Aedh Alreshidi. Covariate-aware transformer architectures for long-horizon demand forecasting in Retail analytics[J]. AIMS Mathematics, 2026, 11(7): 21885-21927. doi: 10.3934/math.2026885

Related Papers:

Abstract

Demand forecasting from a retail perspective affects various facets of the business, including workforce planning, inventory control, and supply chain management, and informs key business strategies. The long-range demand forecasting techniques evaluate various external influences and deal with time series data that often exhibit irregular patterns and complex seasonality, and are often hierarchical. In this work, we proposed a long-range demand forecasting framework that used covariates and was based on transformer architectures with the following key components: multihead self-attention, direct multi-horizon prediction, positional encodings, and categorical embeddings. This framework can model various temporal patterns and was context-aware within the retail system. Having implemented the framework, we benchmarked it against state-of-the-art demand forecasting models on several retail forecasting datasets, as well as other complex time-series, multi-dimensional forecasting datasets. The model empirically demonstrated a 3.1% improvement on the retail forecasting benchmark (achieving a weighted root mean squared scaled error (WRMSSE) of 0.498) and a 1.7% improvement on the benchmark for complex time series datasets (achieving an symmetric mean absolute percentage error (sMAPE) of 12.47) compared to the leading transformer models. Additionally, we provided empirical evidence of enhanced demand forecasting, improved stability and robustness, reduced training time, and improved model throughput. With all of these model features and enhancements combined, we believe this will make retail demand forecasting significantly more feasible and add considerable operational value, particularly for the planning and strategy aspects of retail management.

References

[1]	L. Lei, C. Li, Multi-horizon demand forecasting using hierarchical attention mechanism with learned temporal embeddings, In: Proceedings of the 2025 International Conference on Frontiers Technology in Circuits and Systems (FTCS), IEEE, 2025,108–114. https://doi.org/10.1109/FTCS68006.2025.11405761
[2]	G. Theodoridis, A. Tsadiras, Retail demand forecasting: A multivariate approach and comparison of boosting and deep learning methods, Int. J. Artif. Intell. T., 33 (2024), 2450001. https://doi.org/10.1142/S0218213024500015 doi: 10.1142/S0218213024500015
[3]	S. Lin, W. Lin, W. Wu, F. Zhao, R. Mo, H. Zhang, SegRNN: Segment recurrent neural network for long-term time series forecasting, IEEE Internet Things, 13 (2026), 9861–9871. https://doi.org/10.1109/JIOT.2025.3647705 doi: 10.1109/JIOT.2025.3647705
[4]	B. Lim, S. Ö. Arik, N. Loeff, T. Pfister, Temporal fusion transformers for interpretable multi-horizon time series forecasting‌, Int. J. Forecasting, 37 (2021), 1748–1764. https://doi.org/10.1016/j.ijforecast.2021.03.012 doi: 10.1016/j.ijforecast.2021.03.012
[5]	Z. Kuang, H. Yuan, J. Yang, Y. Wang, J. Bi, J. Zhang, SVBTformer: A decomposition-enhanced hybrid transformer for long-term time series forecasting‌, In: 2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, 2025, 2925–2930. https://doi.org/10.1109/SMC58881.2025.11343684
[6]	P. Schlieper, M. Dombrowski, A. Nguyen, D. Zanca, B. Eskofier, Data-centric benchmarking of neural network architectures for the univariate time series forecasting task‌, Forecasting, 6 (2024), 718–747. https://doi.org/10.3390/forecast6030037 doi: 10.3390/forecast6030037
[7]	D. Li, J. Xin, Deep learning-driven intelligent pricing model in retail: From sales forecasting to dynamic price optimization, Soft Comput., 28 (2024), 12281–12297. https://doi.org/10.1007/s00500-024-09937-z doi: 10.1007/s00500-024-09937-z
[8]	S. Chakraborty, I. Delibasoglu, F. Heintz, Scaling transformers for time series forecasting: Do pretrained large models outperform small-scale alternatives? Artif. Intell. Rev., 59 (2026), 62. https://doi.org/10.1007/s10462-025-11481-7 doi: 10.1007/s10462-025-11481-7
[9]	H. Krupakar, V. A. Kandappan, A review of the long horizon forecasting problem in time series analysis, arXiv Preprint, 2025. Available from: https://arXiv.org/abs/2506.12809
[10]	S. Makridakis, E. Spiliotis, V. Assimakopoulos, The M4 competition: Results, findings, conclusion and way forward, Int. J. Forecasting, 34 (2018), 802–808. https://doi.org/10.1016/j.ijforecast.2018.06.001 doi: 10.1016/j.ijforecast.2018.06.001
[11]	S. Makridakis, E. Spiliotis, V. Assimakopoulos, M5 accuracy competition: Results, findings, and conclusions, Int. J. Forecasting, 38 (2022), 1346–1364. https://doi.org/10.1016/j.ijforecast.2021.11.013 doi: 10.1016/j.ijforecast.2021.11.013
[12]	S. Tehsin, I. M. Nasir, R. Damaševičius, Interpreting CNN for brain Tumor classification using XGrad-CAM, In: Advanced Research in Technologies, Information, Innovation and Sustainability, ARTIIS 2024, Communications in Computer and Information Science, Springer, 2348 (2024). https://doi.org/10.1007/978-3-031-83435-6_21
[13]	S. Tehsin, I. M. Nasir, R. Damaševičius, Explainability of disease classification from medical images using XGrad-CAM, In: International conference on advanced research in technologies, information, innovation and sustainability, Springer, 2024,221–236.
[14]	I. M. Nasir, S. Tehsin, R. Damaševičius, A. Zielonka, M. Woźniak, Explainable cubic attention-based autoencoder for skin cancer classification, In: Artificial Intelligence and Soft Computing, Lecture Notes in Networks and Systems, Springer, 15166 (2025). https://doi.org/10.1007/978-3-031-81596-6_11
[15]	S. Tehsin, I. M. Nasir, R. Damaševičius, Explainable brain tumor segmentation via attention-guided hybrid CNN–transformer–Mamba network, Knowl.-Based Syst., 343 (2026), 116000. https://doi.org/10.1016/j.knosys.2026.116000 doi: 10.1016/j.knosys.2026.116000
[16]	I. M. Nasir, H. Alshaya, S. Tehsin, W. Bouchelligua, Adaptive vision–language transformer for multimodal CNS tumor diagnosis‌, Biomedicines, 13 (2025), 2864. https://doi.org/10.3390/biomedicines13122864 doi: 10.3390/biomedicines13122864
[17]	A. Waqar, Intelligent decision support systems in construction engineering: An artificial intelligence and machine learning approaches‌, Expert Syst. Appl., 249 (2024), 123503. https://doi.org/10.1016/j.eswa.2024.123503 doi: 10.1016/j.eswa.2024.123503
[18]	A. Waqar, I. Othman, N. Shafiq, M. S. Mansoor, Applications of AI in oil and gas projects towards sustainable development: A systematic literature review, Artif. Intell. Rev., 56 (2023), 12771–12798. https://doi.org/10.1007/s10462-023-10467-7 doi: 10.1007/s10462-023-10467-7
[19]	S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., 9 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 doi: 10.1162/neco.1997.9.8.1735
[20]	B. N. Oreshkin, D. Carpov, N. Chapados, Y. Bengio, N-BEATS: Neural basis expansion analysis for interpretable time series forecasting‌, In: International Conference on Learning Representations (ICLR), 2020. Available from: https://openreview.net/forum?id=r1ecqn4YwB.
[21]	C. Challu, K. G. Olivares, B. N. Oreshkin, F. G. Ramirez, M. M. Canseco, A. Dubrawski, NHiTS: Neural hierarchical interpolation for time series forecasting, In: Proceedings of the AAAI Conference on Artificial Intelligence, 37 (2023), 6989–6997. https://doi.org/10.1609/aaai.v37i6.25854
[22]	A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, In: Advances in Neural Information Processing Systems, 30 (2017). Available from: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
[23]	H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, et al., Informer: Beyond efficient transformer for long sequence time-series forecasting, In: Proceedings of the AAAI Conference on Artificial Intelligence, 35 (2021), 11106–11115. https://doi.org/10.1609/aaai.v35i12.17325
[24]	H. Wu, J. Xu, J. Wang, M. Long, Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, In: Advances in Neural Information Processing Systems, 34 (2021), 22419–22430. Available from: https://proceedings.neurips.cc/paper_files/paper/2021/file/bcc0d400288793e8bdcd7c19a8ac0c2b-Paper.pdf.
[25]	Y. Nie, N. H. Nguyen, P. Sinthong, J. Kalagnanam, A time series is worth 64 words: Long-term forecasting with transformers, In: International Conference on Learning Representations (ICLR), 2023. Available from: https://openreview.net/forum?id=Jbdc0vTOcol.
[26]	B. Lim, S. Ö. Arik, N. Loeff, T. Pfister, Temporal fusion transformers for interpretable multi-horizon time series forecasting, Int. J. Forecasting, 37 (2021), 1748–1764. https://doi.org/10.1016/j.ijforecast.2021.03.012 doi: 10.1016/j.ijforecast.2021.03.012
[27]	R. Caetano, J. M. Oliveira, P. Ramos, Transformer-based models for probabilistic time series forecasting with explanatory variables, Mathematics, 13 (2025), 814. https://doi.org/10.3390/math13050814 doi: 10.3390/math13050814
[28]	Y. Yamaguchi, I. Suemitsu, W. Wei, CITRAS: Covariate-informed transformer for time series forecasting, IEEE Access, 14 (2026). https://doi.org/10.1109/ACCESS.2026.3571646 doi: 10.1109/ACCESS.2026.3571646
[29]	L. Han, Y. Liu, Q. Deng, J. Jiang, Y. Sun, Z. Yu, et al., UniCA: Adapting time series foundation model to general covariate-aware forecasting‌, arXiv Preprint, 2025. Available from: https://arXiv.org/abs/2506.22039.
[30]	S. B. Punati, S. Kanta, U. B. Cheerala, M. G. Lanjewar, P. Damacharla, Temporal fusion transformer for multi-horizon probabilistic forecasting of weekly retail sales, arXiv Preprint, 2025. https://doi.org/10.48550/arXiv.2511.00552
[31]	S. Zhao, Y. Lin, X. Yang, Q. Lu, H. Xue, G. Jiang, Optimization of deep learning models for dynamic market behavior prediction, arXiv Preprint, 2025. https://doi.org/10.48550/arXiv.2511.19090
[32]	Z. Lyu, A. Ororbia, T. Desell, Online evolutionary neural architecture search for multivariate non-stationary time series forecasting, Applied Soft Comput., 145 (2023), 110522. https://doi.org/10.1016/j.asoc.2023.110522 doi: 10.1016/j.asoc.2023.110522
[33]	L. Su, X. Zuo, R. Li, X. Wang, H. Zhao, B. Huang, A systematic review for transformer-based long-term series forecasting‌, Artif. Intell. Rev., 58 (2025), 80. https://doi.org/10.1007/s10462-024-11044-2 doi: 10.1007/s10462-024-11044-2
[34]	S. J. Taylor, B. Letham, Forecasting at scale, Am. Stat., 72 (2018), 37–45. https://doi.org/10.1080/00031305.2017.1380080 doi: 10.1080/00031305.2017.1380080
[35]	D. Salinas, V. Flunkert, J. Gasthaus, T. Januschowski, DeepAR: Probabilistic forecasting with autoregressive recurrent networks‌, Int. J. Forecasting, 36 (2020), 1181–1191. https://doi.org/10.1016/j.ijforecast.2019.07.001 doi: 10.1016/j.ijforecast.2019.07.001

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)