Multi-objective pedestrian tracking method based on YOLOv8 and improved DeepSORT

Wenshun Sheng; Jiahui Shen; Qiming Huang; Zhixuan Liu; Zihao Ding; Wenshun Sheng; Jiahui Shen; Qiming Huang; Zhixuan Liu; Zihao Ding

doi:10.3934/mbe.2024077

Mathematical Biosciences and Engineering

2024, Volume 21, Issue 2: 1791-1805. doi: 10.3934/mbe.2024077

Previous Article Next Article

Research article Special Issues

Multi-objective pedestrian tracking method based on YOLOv8 and improved DeepSORT

Pujiang Institute, Nanjing Tech University, Nanjing 211200, China

Academic Editor: Namhyuk Ahn

Received: 26 September 2023 Revised: 05 December 2023 Accepted: 21 December 2023 Published: 03 January 2024

A multi-objective pedestrian tracking method based on you only look once-v8 (YOLOv8) and the improved simple online and real time tracking with a deep association metric (DeepSORT) was proposed with the purpose of coping with the issues of local occlusion and ID dynamic transformation that frequently arise when tracking target pedestrians in real complex traffic scenarios. To begin with, in order to enhance the feature extraction network's capacity to learn target feature information in busy traffic situations, the detector implemented the YOLOv8 method with a high level of small-scale feature expression. In addition, the omni-scale network (OSNet) feature extraction network was then put on top of DeepSORT in order to accomplish real-time synchronized target tracking. This increases the effectiveness of picture edge recognition by dynamically fusing the collected feature information at various scales. Furthermore, a new adaptive forgetting smoothing Kalman filtering algorithm (FSA) was created to adapt to the nonlinear condition of the pedestrian trajectory in the traffic scene in order to address the issue of poor prediction attributed to the linear state equation of Kalman filtering once more. Afterward, the original intersection over union (IOU) association matching algorithm of DeepSORT was replaced by the complete-intersection over union (CIOU) association matching algorithm to fundamentally reduce the target pedestrians' omission and misdetection situation and to improve the accuracy of data matching. Eventually, the generalized trajectory feature extractor model (GFModel) was developed to tightly merge the local and global information through the average pooling operation in order to get precise tracking results and further decrease the impact of numerous disturbances on target tracking. The fusion algorithm of YOLOv8 and improved DeepSORT method based on OSNet, FSA and GFModel was named YOFGD. According to the experimental findings, YOFGD's ultimate accuracy can reach 77.9% and its speed can reach 55.8 frames per second (FPS), which is more than enough to fulfill the demands of real-world scenarios.
- pedestrian tracking,
- YOLOv8,
- DeepSORT,
- multi-objective tracking,
- association matching
Citation: Wenshun Sheng, Jiahui Shen, Qiming Huang, Zhixuan Liu, Zihao Ding. Multi-objective pedestrian tracking method based on YOLOv8 and improved DeepSORT[J]. Mathematical Biosciences and Engineering, 2024, 21(2): 1791-1805. doi: 10.3934/mbe.2024077

Related Papers:

Abstract

A multi-objective pedestrian tracking method based on you only look once-v8 (YOLOv8) and the improved simple online and real time tracking with a deep association metric (DeepSORT) was proposed with the purpose of coping with the issues of local occlusion and ID dynamic transformation that frequently arise when tracking target pedestrians in real complex traffic scenarios. To begin with, in order to enhance the feature extraction network's capacity to learn target feature information in busy traffic situations, the detector implemented the YOLOv8 method with a high level of small-scale feature expression. In addition, the omni-scale network (OSNet) feature extraction network was then put on top of DeepSORT in order to accomplish real-time synchronized target tracking. This increases the effectiveness of picture edge recognition by dynamically fusing the collected feature information at various scales. Furthermore, a new adaptive forgetting smoothing Kalman filtering algorithm (FSA) was created to adapt to the nonlinear condition of the pedestrian trajectory in the traffic scene in order to address the issue of poor prediction attributed to the linear state equation of Kalman filtering once more. Afterward, the original intersection over union (IOU) association matching algorithm of DeepSORT was replaced by the complete-intersection over union (CIOU) association matching algorithm to fundamentally reduce the target pedestrians' omission and misdetection situation and to improve the accuracy of data matching. Eventually, the generalized trajectory feature extractor model (GFModel) was developed to tightly merge the local and global information through the average pooling operation in order to get precise tracking results and further decrease the impact of numerous disturbances on target tracking. The fusion algorithm of YOLOv8 and improved DeepSORT method based on OSNet, FSA and GFModel was named YOFGD. According to the experimental findings, YOFGD's ultimate accuracy can reach 77.9% and its speed can reach 55.8 frames per second (FPS), which is more than enough to fulfill the demands of real-world scenarios.

References

[1]	H. Liu, F. Dong, Multi object detection algorithm under complex traffic conditions based on YOLOv4, Foreign Electron. Meas. Technol., 41 (2022), 41–47. https://doi.org/10.19652/j.cnki.femt.2204351 doi: 10.19652/j.cnki.femt.2204351
[2]	Z. X. Zou, K. Y. Chen, Z. W. Wei, Y. H. Gou, J. P. Ye, Object Detection in 20 Years: A Survey, Proc. IEEE, 111 (2023), 257–276. https://doi.org/10.1109/JPROC.2023.3238524 doi: 10.1109/JPROC.2023.3238524
[3]	F. M. Talaat, H. ZainEldin, An improved fire detection approach based on YOLO-v8 for smart cities, Neural Comput. Appl., 35 (2023), 20939–-20954. https://doi.org/10.1007/s00521-023-08809-1 doi: 10.1007/s00521-023-08809-1
[4]	M. I. H. Azhar, F. H. K. Zaman, N. M. Tahir, H. Hashim, People tracking system using DeepSORT, in 2020 10th IEEE International Conference on Control System, Computing and Engineering (ICCSCE), Penang, Malaysia, (2022), 137–141. https://doi.org/10.1109/ICCSCE50387.2020.9204956
[5]	G. Yang, Z. Chen, Pedestrian tracking algorithm for dense crowd based on deep learning, in Proceedings of 2019 6th International Conference on Systems and Informatics (ICSAI), (2019), 568–572. https://doi.org/https://doi.org/10.1109/ICSAI48974.2019.9010144
[6]	M. I. H. Azhar, F. H. K. Zaman, N. M. Tahir, H. Hashim, People tracking system using DeepSORT, in Proceedings of 2020 10th IEEE International Conference on Control System, Computing and Engineering (ICCSCE), (2020), 137–141. https://doi.org/https://doi.org/10.1109/ICCSCE50387.2020.9204956
[7]	D. Stadler, J. Beyerer, Multi-Pedestrian tracking with clusters, in Proceedings of 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), (2021), 1–10. https://doi.org/https://doi.org/10.1109/AVSS52988.2021.9663829
[8]	C. Li, Y. Wang, X. Liu, A Multi-Pedestrian tracking algorithm for dense scenes based on an attention mechanism and dual data association, Appl. Sci., 12 (2022), 9597. https://doi.org/10.3390/app12199597 doi: 10.3390/app12199597
[9]	Q. Gao, Z. He, X. Jia, Y. Xie, X. Han, Lightweight high-precision pedestrian tracking algorithm in complex occlusion scenarios, KSII Trans. Int. Inform. Syst., 17 (2023), 840–860. https://doi.org/10.3837/tiis.2023.03.009 doi: 10.3837/tiis.2023.03.009
[10]	R. A. Zitar, A. Mohsen, A. E. Seghrouchni, F. Barbaresco, N. A. Al-Dmour, Intensive review of drones detection and tracking: Linear Kalman filter versus nonlinear regression, an analysis case, Arch Comput. Methods Eng., 14 (2023), 2811-–2830. https://doi.org/10.1007/s11831-023-09894-0 doi: 10.1007/s11831-023-09894-0
[11]	X. B. Liu, X. Z. Yang, Y. Chen, S. T. Zhao, Object detection method based on CIoU improved bounding box loss function, Chinese J. Liquid Cryst. Displ., 38 (2023), 656–665. https://doi.org/10.37188/CJLCD.2022-0282 doi: 10.37188/CJLCD.2022-0282
[12]	C. Y. Wang, H. Y. M. Liao, I. H. Yeh, Designing network design strategies through gradient path analysis, preprint, arXiv: 2211.04800.
[13]	H. T. Liu, X. H. Duan, J. M. Guo, H. Y. Liu, J. Gu, H. Chen, DC-YOLOv8: Small-Size object detection algorithm based on camera sensor, Electronics, 12 (2023), 2323. https://doi.org/10.3390/electronics12102323 doi: 10.3390/electronics12102323
[14]	H. Z. Xu, H. J. He, Y. Zhang, L. F. Ma, J. T. Li, A comparative study of loss functions for road segmentation in remotely sensed road datasets, Int. J. Appl. Earth Observ. Geoinform., 116 (2023), 1569–8432. https://doi.org/10.1016/j.jag.2022.103159 doi: 10.1016/j.jag.2022.103159
[15]	M. S. Hossain, J. M. Betts, A. P. Paplinski, Dual Focal Loss to address class imbalance in semantic segmentation, Neurocomputing, 462 (2021), 69–87. https://doi.org/10.1016/j.neucom.2021.07.055 doi: 10.1016/j.neucom.2021.07.055
[16]	I. Pacal, D. Karaboga, A robust real-time deep learning based automatic polyp detection system, Comput. Biol. Med., 134 (2021), 104519. https://doi.org/10.1016/j.compbiomed.2021.104519 doi: 10.1016/j.compbiomed.2021.104519
[17]	A. X. Zhao, J. Q. Yang, H. B. Yang, X. G. Shi, W. X. Fu, S. Liu, et al., Indoor multi-object personnel recognition and tracking across camera based on optimized DeepSORT and FastReID, J. Xi'an Univ. Sci. Technol., 43 (2023), 620–630. https://doi.org/10.13800/j.cnki.xakjdxxb.2023.0320 doi: 10.13800/j.cnki.xakjdxxb.2023.0320
[18]	T. Jin, X. Ye, Z. Li, Z. Huo, Identification and tracking of vehicles between multiple cameras on bridges using a YOLOv4 and OSNet-Based method, Sensors, 23 (2023), 5510. https://doi.org/10.1109/ICSPIS56952.2022.10043932 doi: 10.1109/ICSPIS56952.2022.10043932
[19]	C. Nie, Z. Ju, Z. Sun, H. Zhang, 3D object detection and tracking based on Lidar-Camera fusion and IMM-UKF algorithm towards highway driving, in IEEE Trans. Emerging Topics Comput. Intell., 7 (2023), 1242–1252. https://doi.org/10.1109/TETCI.2023.3259441
[20]	S. Mittal, A Survey on optimized implementation of deep learning models on the NVIDIA Jetson platform, J. Syst. Arch., 97 (2019), 428–442. https://doi.org/10.1016/j.sysarc.2019.01.011 doi: 10.1016/j.sysarc.2019.01.011
[21]	S. Uladzislaum, X. Feng, Modified omni-scale net architecture for cattle identification on their muzzle point image pattern characteristics, in International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 12645 (2023), 489–494. https://doi.org/10.1117/12.2681201
[22]	H. Xie, Z. Xiao, W. Liu, Z. Ye, PVNet: A Used Vehicle Pedestrian Detection Tracking and Counting Method, Sustainability, 15 (2023), 14326. https://doi.org/10.3390/su151914326 doi: 10.3390/su151914326
[23]	R. P. Tripathi, A. K. Singh, P. Gangwar, Fractional order adaptive Kalman filter for sensorless speed control of DC motor, Int. J. Electron., 110 (2023), 373–390. http://dx.doi.org/10.5081/jgps.2.1.42 doi: 10.5081/jgps.2.1.42
[24]	N. Wanchaitanawong, M. Tanaka, T. Shibata, M. Okutomi, Multi-modal pedestrian detection with misalignment based on modal-wise regression and multi-modal IoU, J. Electron. Imaging, 32 (2023), 013025. https://doi.org/10.1117/1.JEI.32.1.013025 doi: 10.1117/1.JEI.32.1.013025
[25]	N. K. S. Behera, P. K. Sa, S. Bakshi, U. Bilotti, Explainable graph-attention based person re-identification in outdoor conditions, Multimed Tools Appl., 2023 (2023), 99–108. https://doi.org/10.1007/s11042-023-16986-3 doi: 10.1007/s11042-023-16986-3
[26]	M. K. Vidhyalakshmi, E. Poovammal, V. Bhaskar, J. Sathyanarayanan, Novel similarity metric learning using deep learning and root SIFT for person re-identification, Wireless Personal Commun., 117 (2021), 1835–1851. https://doi.org/10.1007/s11277-020-07948-1 doi: 10.1007/s11277-020-07948-1
[27]	O. Tomarchio, D. Calcaterra, G. D. Modica, P. Mazzaglia, Torch: a tosca-based orchestrator of multi-cloud containerised applications, J. Grid Comput., 19 (2021), 1–25. https://doi.org/10.1007/s10723-021-09549-z doi: 10.1007/s10723-021-09549-z
[28]	M. Razzok, A. Badri, I. E. Mourabit, Y. Ruichek, A. Sahel, Pedestrian detection and tracking system based on Deep-SORT, YOLOv5, and new data association metrics, Information, 14 (2023), 218. https://doi.org/10.3390/info14040218 doi: 10.3390/info14040218
[29]	X. Xiao, X. Feng, Multi-Object pedestrian tracking using improved YOLOv8 and OC-SORT, Sensors, 23 (2023), 8439. https://doi.org/10.3390/s23208439 doi: 10.3390/s23208439

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)