To assure operational safety in the airport apron area and track the process of ground service, it is necessary to analyze key targets and their activities in the airport apron surveillance videos. This research shows an activity identification algorithm for ground service objects in an airport apron area and proposes an improved YOLOv5 algorithm to increase the precision of small object detection by introducing an SPD-Conv (spath-to-depth-Conv) block in YOLOv5's backbone layer. The improved algorithm can efficiently extract the information features of small-sized objects, medium-sized objects, and moving objects in large scenes, and it achieves effective detection of activities of ground service in the apron area. The experimental results show that the detection average precision of all objects is more than 90%, and the whole class mean average precision (mAP) is 98.7%. At the same time, the original model was converted to TensorRT and OpenVINO format models, which increased the inference efficiency of the GPU and CPU by 55.3 and 137.1%, respectively.
Citation: Yaxi Xu, Yi Liu, Ke Shi, Xin Wang, Yi Li, Jizong Chen. An airport apron ground service surveillance algorithm based on improved YOLO network[J]. Electronic Research Archive, 2024, 32(5): 3569-3587. doi: 10.3934/era.2024164
To assure operational safety in the airport apron area and track the process of ground service, it is necessary to analyze key targets and their activities in the airport apron surveillance videos. This research shows an activity identification algorithm for ground service objects in an airport apron area and proposes an improved YOLOv5 algorithm to increase the precision of small object detection by introducing an SPD-Conv (spath-to-depth-Conv) block in YOLOv5's backbone layer. The improved algorithm can efficiently extract the information features of small-sized objects, medium-sized objects, and moving objects in large scenes, and it achieves effective detection of activities of ground service in the apron area. The experimental results show that the detection average precision of all objects is more than 90%, and the whole class mean average precision (mAP) is 98.7%. At the same time, the original model was converted to TensorRT and OpenVINO format models, which increased the inference efficiency of the GPU and CPU by 55.3 and 137.1%, respectively.
| [1] | Civil Aviation Administration of China, CAAC Issues 2022 Statistical Bulletin of Civil Airport Production in China, 2023. Available from: https://www.caac.gov.cn/XXGK/XXGK/TJSJ/202303/t20230317_217609.html. |
| [2] | J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 779–788. https://doi.org/10.1109/CVPR.2016.91 |
| [3] | H. Law, J. Deng, Cornernet: Detecting objects as paired keypoints, in Proceedings of the European Conference on Computer Vision (ECCV), (2018), 734–750. https://doi.org/10.1007/s11263-019-01204-1 |
| [4] | Z. Tian, C. Shen, H. Chen, T. He, Fcos: Fully convolutional one-stage object detection, in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019), 9626–9636. https://doi.org/10.1109/ICCV.2019.00972 |
| [5] |
J. Liu, M. Wang, X. Xie, Y. Song, L. Xu, Leather defect detection algorithm based on improved YOLOv5, Comput. Eng., 49 (2023), 240–249. https://doi.org/10.19678/j.issn.1000-3428.0064587 doi: 10.19678/j.issn.1000-3428.0064587
|
| [6] | J. Choi, D. Chun, H. Kim, H. J. Lee, Gaussian yolov3: An accurate and fast object detector using localization uncertainty for autonomous driving, in Proceedings of the IEEE/CVF International Conference on Computer Vision, (2019), 502–511. https://doi.org/10.1109/ICCV.2019.00059 |
| [7] | P. Li, H. Li, Research on fod detection for airport runway based on yolov3, in 2020 39th Chinese Control Conference (CCC), IEEE, (2020), 7096–7099. https://doi.org/10.23919/CCC50068.2020.9188724 |
| [8] | X. Shi, J. Hu, X. Lei, S. Xu, Detection of flying birds in airport monitoring based on improved YOLOv5, in 2021 6th International Conference on Intelligent Computing and Signal Processing (ICSP), IEEE, (2021), 1446–1451. https://doi.org/10.1109/ICSP51882.2021.9408797 |
| [9] |
F. H. Cai, Y. X. Zhang, J. Huang, A bridge surface crack detection algorithm based on YOLOv3 and attention mechanism, Pattern Recognit. Artif. Intell., 33 (2020), 926–933. https://doi.org/10.16451/j.cnki.issn1003-6059.202010007 doi: 10.16451/j.cnki.issn1003-6059.202010007
|
| [10] | Y. M. Shi, K. B. Jia, The Research and implementation of moving object detecting and tracking in intelligence video monitor system, in 2011 International Conference on Multimedia and Signal Processing, IEEE, 2 (2011), 105–108. https://doi.org/10.1109/CMSP.2011.111 |
| [11] | J. Ferryman, M. Borg, D. Thirde, F. Fusier, V. Valentin, F. Brémond, et al., Automated scene understanding for airport aprons, in AI 2005: Advances in Artificial Intelligence. AI 2005. Lecture Notes in Computer Science, Springer, Berlin, 3809 (2005), 593–603. https://doi.org/10.1007/11589990_62 |
| [12] |
N. Cai, G. He, Multi-cloud resource scheduling intelligent system with endogenous security, Electron. Res. Arch., 32 (2024), 1380–1405. https://doi/10.3934/era.2024064 doi: 10.3934/era.2024064
|
| [13] |
M. Kumar, S. Ray, D. K. Yadav, Moving human detection and tracking from thermal video through intelligent surveillance system for smart applications, Multimedia Tools Appl., 82 (2023), 39551–39570. https://doi.org/10.1007/s11042-022-13515-6 doi: 10.1007/s11042-022-13515-6
|
| [14] |
A. Raza, S. A. Chelloug, M. H. Alatiyyah, A. Jalal, J. Park, Multiple pedestrian detection and tracking in night vision surveillance systems, CMC-Comput. Mater. Continua, 75 (2023), 3275–3289. http://doi.org/10.32604/cmc.2023.029719 doi: 10.32604/cmc.2023.029719
|
| [15] |
S. Wilke, A. Majumdar, W. Y. Ochieng, Airport surface operations: A holistic framework for operations modeling and risk management, Saf. Sci., 63 (2014), 18–33. https://doi.org/10.1016/j.ssci.2013.10.015 doi: 10.1016/j.ssci.2013.10.015
|
| [16] | H. L. Lu, S. Vaddi, V. Cheng, J. Tsai, Airport gate operation monitoring using computer vision techniques, in 16th AIAA Aviation Technology, Integration, and Operations Conference, (2016), 3912. https://doi.org/10.2514/6.2016-3912 |
| [17] | Y. Zou, Q. Ying, R. Liu, M. Rong, Research on evaluation method for operation support capability of airport apron, in 2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT, IEEE, (2020), 1043–1047. https://doi.org/10.1109/ICCASIT50869.2020.9368541 |
| [18] | T. V. Phat, S. Alam, N. Lilith, P. N. Tran, B. T. Nguyen, Aircraft push-back prediction and turnaround monitoring by vision-based object detection and activity identification, in Proc. 10th SESAR Innov. Days., 2020. |
| [19] | M. Gorkow, Aircraft Turnaround Management Using Computer Vision, 2020. Available from: https://medium.com/@michaelgorkow/aircraft-turnaround-management-using-computer-vision-4bec29838c08. |
| [20] |
S. Wang, Y. Che, H. Zhao, A. Lim, Accurate tracking, collision detection, and optimal scheduling of airport ground support equipment, IEEE Internet Things J., 8 (2020), 572–584. https://doi.org/10.1109/JIOT.2020.3004874 doi: 10.1109/JIOT.2020.3004874
|
| [21] |
S. Wang, C. Li, A. Lim, ROPHS: Determine real-time status of a multi-carriage logistics train at airport, IEEE Trans. Intell. Transp. Syst., 23 (2021), 6347–6356. https://doi.org/10.1109/TITS.2021.3055838 doi: 10.1109/TITS.2021.3055838
|
| [22] |
S. Yıldız, O. Aydemir, A. Memiş, S. Varlı, A turnaround control system to automatically detect and monitor the time stamps of ground service actions in airports: a deep learning and computer vision based approach, Eng. Appl. Artif. Intell., 114 (2022), 105032. https://doi.org/10.1016/j.engappai.2022.105032 doi: 10.1016/j.engappai.2022.105032
|
| [23] |
P. Thai, S. Alam, N. Lilith, B. T. Nguyen, A computer vision framework using Convolutional Neural Networks for airport-airside surveillance, Transp. Res. Part C Emerging Technol., 137 (2022), 103590. https://doi.org/10.1016/j.trc.2022.103590 doi: 10.1016/j.trc.2022.103590
|
| [24] | S. Liu, R. Wu, J. Qu, Y. Li, HDA-Net: hybrid convolutional neural networks for small objects recognization at airports, IEEE Trans. Instrum. Meas., 71 (2022), 1–14. https://ieeexplore.ieee.org/abstract/document/9939036 |
| [25] | MH/T 6125—2022, Technical specifications for airport collaborative decision making system, 2022. Available from: https://www.caac.gov.cn/XXGK/XXGK/BZGF/HYBZ/202202/P020220228396026654632.pdf. |
| [26] | R. Sunkara, T. Luo, No more strided convolutions or pooling: A new CNN building block for low-resolution images and small objects, in Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Cham, Springer Nature Switzerland, (2022), 443–459. https://doi.org/10.1007/978-3-031-26409-2_27 |
| [27] | S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: Towards real-time object detection with region proposal networks, Adv. Neural Inf. Process. Syst., (2015), 28. |
| [28] | W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, et al., Ssd: Single shot multibox detector, in Computer Vision–ECCV 2016. ECCV 2016. Lecture Notes in Computer Science(), Springer, Cham, 9905 (2016), 21–37 https://doi.org/10.1007/978-3-319-46448-0_2 |
| [29] | M. Tan, R. Pang, Q. V. Le, Efficientdet: Scalable and efficient object detection, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020), 10781–10790. |
| [30] | K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, et al., MMDetection: Open mmlab detection toolbox and benchmark, preprint, arXiv: 1906.07155. https://doi.org/10.48550/arXiv.1906.07155 |
Figures(11) / Tables(5)
Yaxi Xu, Yi Liu, Ke Shi, Xin Wang, Yi Li, Jizong Chen. An airport apron ground service surveillance algorithm based on improved YOLO network[J]. Electronic Research Archive, 2024, 32(5): 3569-3587. doi: 10.3934/era.2024164
DownLoad: