An infrared small target detection model via Gather-Excite attention and normalized Wasserstein distance

Kangjian Sun; Ju Huo; Qi Liu; Shunyuan Yang; Kangjian Sun; Ju Huo; Qi Liu; Shunyuan Yang

doi:10.3934/mbe.2023842

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 11: 19040-19064. doi: 10.3934/mbe.2023842

Previous Article Next Article

Research article Special Issues

An infrared small target detection model via Gather-Excite attention and normalized Wasserstein distance

1.
School of Electrical Engineering and Automation, Harbin Institute of Technology, Harbin 150001, China
2.
School of Astronautics, Harbin Institute of Technology, Harbin 150001, China

Academic Editor: Yiu-ming Cheung

Received: 16 July 2023 Revised: 11 September 2023 Accepted: 28 September 2023 Published: 11 October 2023

Infrared small target detection (ISTD) is the main research content for defense confrontation, long-range precision strikes and battlefield intelligence reconnaissance. Targets from the aerial view have the characteristics of small size and dim signal. These characteristics affect the performance of traditional detection models. At present, the target detection model based on deep learning has made huge advances. The You Only Look Once (YOLO) series is a classic branch. In this paper, a model with better adaptation capabilities, namely ISTD-YOLOv7, is proposed for infrared small target detection. First, the anchors of YOLOv7 are updated to provide prior. Second, Gather-Excite (GE) attention is embedded in YOLOv7 to exploit feature context and spatial location information. Finally, Normalized Wasserstein Distance (NWD) replaces IoU in the loss function to alleviate the sensitivity of YOLOv7 for location deviations of small targets. Experiments on a standard dataset show that the proposed model has stronger detection performance than YOLOv3, YOLOv5s, SSD, CenterNet, FCOS, YOLOXs, DETR and the baseline model, with a mean Average Precision (mAP) of 98.43%. Moreover, ablation studies indicate the effectiveness of the improved components.
- infrared small target detection,
- YOLOv7,
- anchor update,
- Gather-Excite attention,
- normalized Wasserstein distance
Citation: Kangjian Sun, Ju Huo, Qi Liu, Shunyuan Yang. An infrared small target detection model via Gather-Excite attention and normalized Wasserstein distance[J]. Mathematical Biosciences and Engineering, 2023, 20(11): 19040-19064. doi: 10.3934/mbe.2023842

Related Papers:

Abstract

Infrared small target detection (ISTD) is the main research content for defense confrontation, long-range precision strikes and battlefield intelligence reconnaissance. Targets from the aerial view have the characteristics of small size and dim signal. These characteristics affect the performance of traditional detection models. At present, the target detection model based on deep learning has made huge advances. The You Only Look Once (YOLO) series is a classic branch. In this paper, a model with better adaptation capabilities, namely ISTD-YOLOv7, is proposed for infrared small target detection. First, the anchors of YOLOv7 are updated to provide prior. Second, Gather-Excite (GE) attention is embedded in YOLOv7 to exploit feature context and spatial location information. Finally, Normalized Wasserstein Distance (NWD) replaces IoU in the loss function to alleviate the sensitivity of YOLOv7 for location deviations of small targets. Experiments on a standard dataset show that the proposed model has stronger detection performance than YOLOv3, YOLOv5s, SSD, CenterNet, FCOS, YOLOXs, DETR and the baseline model, with a mean Average Precision (mAP) of 98.43%. Moreover, ablation studies indicate the effectiveness of the improved components.

References

[1]	B. Jiang, X. Ma, Y. Lu, Y. Li, L. Feng, Z. Shi, Ship detection in spaceborne infrared images based on Convolutional Neural Networks and synthetic targets, Infrared Phys. Technol., 97 (2019), 229–234. https://doi.org/10.1016/j.infrared.2018.12.040 doi: 10.1016/j.infrared.2018.12.040
[2]	A. Özdil, B. Yılmaz, Automatic body part and pose detection in medical infrared thermal images, Quant. InfraRed Thermogr. J., 19 (2021), 223–238. https://doi.org/10.1080/17686733.2021.1947595 doi: 10.1080/17686733.2021.1947595
[3]	F. Prata, Detection and avoidance of atmospheric aviation hazards using infrared spectroscopic imaging, Remote Sens., 12 (2020), 2309. https://doi.org/10.3390/rs12142309 doi: 10.3390/rs12142309
[4]	C. Gao, L. Wang, Y. Xiao, Q. Zhao, D. Meng, Infrared small-dim target detection based on Markov random field guided noise modelling, Pattern Recognit., 76 (2018), 463–475. https://doi.org/10.1016/j.patcog.2017.11.016 doi: 10.1016/j.patcog.2017.11.016
[5]	M. Qi, L. Liu, S. Zhuang, Y. Liu, K. Li, Y. Yang, et al., FTC-Net: Fusion of transformer and CNN features for infrared small target detection, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 15 (2022), 8613–8623. https://doi.org/10.1109/JSTARS.2022.3210707 doi: 10.1109/JSTARS.2022.3210707
[6]	N. Nguyen, T. Do, T. Ngo, D. Le, An evaluation of deep learning methods for small object detection, J. Electr. Comput. Eng., 2020 (2020), 3189691. https://doi.org/10.1155/2020/3189691 doi: 10.1155/2020/3189691
[7]	R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, (2014), 580–587. https://doi.org/10.1109/CVPR.2014.81
[8]	J. Li, X. Liang, S. Shen, T. Xu, J. Feng, S. Yan, Scale-aware fast R-CNN for pedestrian detection, IEEE Trans. Multimedia, 20 (2017), 985–996. https://doi.org/10.1109/TMM.2017.2759508 doi: 10.1109/TMM.2017.2759508
[9]	S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031 doi: 10.1109/TPAMI.2016.2577031
[10]	K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask R-CNN, IEEE Trans. Pattern Anal. Mach. Intell., 42 (2020), 386–397. https://doi.org/10.1109/TPAMI.2018.2844175 doi: 10.1109/TPAMI.2018.2844175
[11]	P. Jiang, D. Ergu, F. Liu, Y. Cai, B. Ma, A review of YOLO algorithm developments, Procedia Comput. Sci., 199 (2022), 1066–1073. https://doi.org/10.1016/j.procs.2022.01.135 doi: 10.1016/j.procs.2022.01.135
[12]	J. Redmon, A. Farhadi, YOLOv3: An incremental improvement, preprint, arXiv: 1804.02767.
[13]	S. Shen, X. Zhang, W. Yan, S. Xie, B. Yu, S. Wang, An improved UAV target detection algorithm based on ASFF-YOLOv5s, Math. Biosci. Eng., 20 (2023), 10773–10789. https://doi.org/10.3934/mbe.2023478 doi: 10.3934/mbe.2023478
[14]	Z. Ge, S. Liu, F. Wang, Z. Li, J. Sun, YOLOX: Exceeding YOLO series in 2021, preprint, arXiv: 2107.08430.
[15]	C. Wang, A. Boschkovskiy, H. Liao, YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors, preprint, arXiv: 2207.0269.
[16]	M. Soeb, M. Jubayer, T. Tarin, M. Mamun, F. Ruhad, A. Parven, et al., Tea leaf disease detection and identification based on YOLOv7 (YOLO-T), Sci. Rep., 13 (2023), 6078. https://doi.org/10.1038/s41598-023-33270-4 doi: 10.1038/s41598-023-33270-4
[17]	S. Li, J. Yu, H. Wang, Damages detection of aeroengine blades via deep learning algorithms, IEEE Trans. Instrum. Meas., 72 (2023), 1–11. https://doi.org/10.1109/TIM.2023.3249247 doi: 10.1109/TIM.2023.3249247
[18]	S. Liu, Y. Wang, Q. Yu, H. Liu, Z. Peng, CEAM-YOLOv7: Improved YOLOv7 based on channel expansion and attention mechanism for driver distraction behavior detection, IEEE Access, 10 (2022), 129116–129124. https://doi.org/10.1109/ACCESS.2022.3228331 doi: 10.1109/ACCESS.2022.3228331
[19]	F. Chen, C. Gao, F. Liu, Y. Zhao, Y. Zhou, D. Meng, et al., Local patch network with global attention for infrared small target detection, IEEE Trans. Aerosp. Electron. Syst., 58 (2022), 3979–3991. https://doi.org/10.1109/TAES.2022.3159308 doi: 10.1109/TAES.2022.3159308
[20]	Y. Dai, Y. Wu, F. Zhou, K. Barnard, Attentional local contrast networks for infrared small target detection, IEEE Trans. Geosci. Remote Sens., 59 (2021), 9813–9824. https://doi.org/10.1109/TGRS.2020.3044958 doi: 10.1109/TGRS.2020.3044958
[21]	M. Zhang, R. Zhang, Y. Yang, H. Bai, J. Zhang, J. Guo, ISNet: Shape matters for infrared small target detection, in Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2022), 867–876. https://doi.org/10.1109/CVPR52688.2022.00095
[22]	B. Li, C. Xiao, L. Wang, Y. Wang, Z. Lin, M. Li, W. An, et al., Dense nested attention network for infrared small target detection, IEEE Trans. Image Process., 32 (2023), 1745–1758. https://doi.org/10.1109/TIP.2022.3199107 doi: 10.1109/TIP.2022.3199107
[23]	T. Wu, B. Li, Y. Luo, Y. Wang, C. Xiao, T. Liu, et al., MTU-Net: Multilevel TransUNet for space-based infrared tiny ship detection, IEEE Trans. Geosci. Remote Sens., 61 (2023), 1–15, Art no. 5601015. https://doi.org/10.1109/TGRS.2023.3235002 doi: 10.1109/TGRS.2023.3235002
[24]	Z. Lin, B. Li, M. Li, L. Wang, T. Wu, Y. Luo, et al., Light-weight infrared small target detection combining cross-scale feature fusion with bottleneck attention module, J. Infrared Millimeter Waves, 41 (2022), 1102–1112. https://doi.org/10.11972/j.issn.1001-9014.2022.06.020 doi: 10.11972/j.issn.1001-9014.2022.06.020
[25]	Y. Liu, X. Wang, SAR ship detection based on improved YOLOv7-Tiny, in Proceedings of the 2022 IEEE 8th International Conference on Computer and Communications, (2022), 2166–2170. https://doi.org/10.1109/ICCC56324.2022.10065775
[26]	Y. Guo, S. Chen, R. Zhan, W. Wang, J. Zhang, LMSD-YOLO: A lightweight YOLO algorithm for multi-scale SAR ship detection, Remote Sens., 14 (2022), 4801. https://doi.org/10.3390/rs14194801 doi: 10.3390/rs14194801
[27]	X. Zhou, L. Jiang, C. Hu, S. Lei, T. Zhang, X. Mou, YOLO-SASE: An improved YOLO algorithm for the small targets detection in complex backgrounds, Sensors, 22 (2022), 4600. https://doi.org/10.3390/s22124600 doi: 10.3390/s22124600
[28]	VOC dataset, Available from: http://host.robots.ox.ac.uk/pascal/VOC/voc2007/.
[29]	COCO dataset, Available from: http://cocodataset.org/#download.
[30]	J. Hu, L. Shen, S. Albanie, G. Sun, A. Vedaldi, Gather-Excite: Exploiting feature context in convolutional neural networks, in Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS'18), (2018), 9423–9433.
[31]	J. Wang, C. Xu, W. Yang, L. Yu, A normalized Gaussian Wasserstein distance for tiny object detection, preprint, arXiv: 2110.13389.
[32]	C. Xu, J. Wang, W. Yang, H. Yu, L. Yu, G. Xia, Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark, ISPRS J. Photogramm. Remote Sens., 190 (2022), 79–93. https://doi.org/10.1016/j.isprsjprs.2022.06.002 doi: 10.1016/j.isprsjprs.2022.06.002
[33]	H. Lai, L. Chen, W. Liu, Z. Yan, S. Ye, STC-YOLO: Small object detection network for traffic signs in complex environments, Sensors, 23 (2023), 5307. https://doi.org/10.3390/s23115307 doi: 10.3390/s23115307
[34]	Z. Zheng, N. Chen, J. Wu, Z. Xv, S. Liu, Z. Luo, EW-YOLOv7: A lightweight and effective detection model for small defects in electrowetting display, Processes, 11 (2023), 2037. https://doi.org/10.3390/pr11072037 doi: 10.3390/pr11072037
[35]	J. Hosang, R. Benenson, B. Schiele, Learning non-maximum suppression, in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 6469–6477. https://doi.org/10.1109/CVPR.2017.685
[36]	R. Fu, H. Fan, Y. Zhu, B. Hui, Z. Zhang, P. Zhong, et al., A dataset for infrared time-sensitive target detection and tracking for air-ground application, China Sci. Data, 7 (2022), 206–221. https://doi.org/10.11922/sciencedb.j00001.00331 doi: 10.11922/sciencedb.j00001.00331
[37]	C. Chen, G. Yuan, H. Zhou, Y. Ma, Improved YOLOv5s model for key components detection of power transmission lines, Math. Biosci. Eng., 20 (2023), 7738–7760. https://doi.org/10.3934/mbe.2023334 doi: 10.3934/mbe.2023334
[38]	M. Huang, Y. Wu, GCS-YOLOV4-Tiny: A lightweight group convolution network for multi-stage fruit detection, Math. Biosci. Eng., 20 (2023), 241–268. https://doi.org/10.3934/mbe.2023011 doi: 10.3934/mbe.2023011
[39]	W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, et al., SSD: Single Shot MultiBox Detector, in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, (2016), 21–37. https://doi.org/10.1007/978-3-319-46448-0_2
[40]	K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, Q. Tian, CenterNet: Keypoint triplets for object detection, in Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019), 6568–6577. https://doi.org/10.1109/ICCV.2019.00667
[41]	Z. Tian, C. Shen, H. Chen, T. He, FCOS: Fully convolutional one-stage object detection, in Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (2019), 9626–9635. https://doi.org/10.1109/ICCV.2019.00972
[42]	N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, S. Zagoruyko, End-to-end object detection with transformers, in Proceedings of the Computer Vision—ECCV 2020, (2020), 213–229. https://doi.org/10.1007/978-3-030-58452-8_13
[43]	I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., Generative adversarial networks, preprint, arXiv: 1406.2661.

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)