Combination of UAV and Raspberry Pi 4B: Airspace detection of red imported fire ant nests using an improved YOLOv4 model

Xiaotang Liu; Zheng Xing; Huanai Liu; Hongxing Peng; Huiming Xu; Jingqi Yuan; Zhiyu Gou; Xiaotang Liu; Zheng Xing; Huanai Liu; Hongxing Peng; Huiming Xu; Jingqi Yuan; Zhiyu Gou

doi:10.3934/mbe.2022634

Mathematical Biosciences and Engineering

2022, Volume 19, Issue 12: 13582-13606. doi: 10.3934/mbe.2022634

Previous Article Next Article

Research article

Combination of UAV and Raspberry Pi 4B: Airspace detection of red imported fire ant nests using an improved YOLOv4 model

1.
College of Materials and Energy, South China Agricultural University, Guangzhou 510642, China
2.
College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
3.
School of Chemistry and Chemical Engineering, South China Technology of University, Guangzhou 510641, China
4.
Key Laboratory of Smart Agricultural Technology in Tropical South China, Ministry of Agriculture and Rural Affairs, Guangzhou 510642, China
5.
Guangzhou Key Laboratory of Intelligent Agriculture, Guangzhou 510642, China

Academic Editor: Yang Kuang

Received: 20 June 2022 Revised: 27 August 2022 Accepted: 31 August 2022 Published: 15 September 2022

Red imported fire ants (RIFA) are an alien invasive pest that can cause serious ecosystem damage. Timely detection, location and elimination of RIFA nests can further control the spread of RIFA. In order to accurately locate the RIFA nests, this paper proposes an improved deep learning method of YOLOv4. The specific methods were as follows: 1) We improved GhostBottleNeck (GBN) and replaced the original CSP block of YOLOv4, so as to compress the network scale and reduce the consumption of computing resources. 2) An Efficient Channel Attention (ECA) mechanism was introduced into GBN to enhance the feature extraction ability of the model. 3) We used Equalized Focal Loss to reduce the loss value of background noise. 4) We increased and improved the upsampling operation of YOLOv4 to enhance the understanding of multi-layer semantic features to the whole network. 5) CutMix was added in the model training process to improve the model's ability to identify occluded objects. The parameters of improved YOLOv4 were greatly reduced, and the abilities to locate and extract edge features were enhanced. Meanwhile, we used an unmanned aerial vehicle (UAV) to collect images of RIFA nests with different heights and scenes, and we made the RIFA nests (RIFAN) airspace dataset. On the RIFAN dataset, through qualitative analysis of the evaluation indicators, mean average precision (MAP) of the improved YOLOv4 model reaches 99.26%, which is 5.9% higher than the original algorithm. Moreover, compared with Faster R-CNN, SSD and other algorithms, improved YOLOv4 has achieved excellent results. Finally, we transplanted the model to the embedded device Raspberry Pi 4B and assembled it on the UAV, using the model's lightweight and high-efficiency features to achieve flexible and fast flight detection of RIFA nests.
- RIFA nests,
- deep learning,
- improved YOLOv4,
- data augmentation,
- UAV
Citation: Xiaotang Liu, Zheng Xing, Huanai Liu, Hongxing Peng, Huiming Xu, Jingqi Yuan, Zhiyu Gou. Combination of UAV and Raspberry Pi 4B: Airspace detection of red imported fire ant nests using an improved YOLOv4 model[J]. Mathematical Biosciences and Engineering, 2022, 19(12): 13582-13606. doi: 10.3934/mbe.2022634

Related Papers:

Abstract

Red imported fire ants (RIFA) are an alien invasive pest that can cause serious ecosystem damage. Timely detection, location and elimination of RIFA nests can further control the spread of RIFA. In order to accurately locate the RIFA nests, this paper proposes an improved deep learning method of YOLOv4. The specific methods were as follows: 1) We improved GhostBottleNeck (GBN) and replaced the original CSP block of YOLOv4, so as to compress the network scale and reduce the consumption of computing resources. 2) An Efficient Channel Attention (ECA) mechanism was introduced into GBN to enhance the feature extraction ability of the model. 3) We used Equalized Focal Loss to reduce the loss value of background noise. 4) We increased and improved the upsampling operation of YOLOv4 to enhance the understanding of multi-layer semantic features to the whole network. 5) CutMix was added in the model training process to improve the model's ability to identify occluded objects. The parameters of improved YOLOv4 were greatly reduced, and the abilities to locate and extract edge features were enhanced. Meanwhile, we used an unmanned aerial vehicle (UAV) to collect images of RIFA nests with different heights and scenes, and we made the RIFA nests (RIFAN) airspace dataset. On the RIFAN dataset, through qualitative analysis of the evaluation indicators, mean average precision (MAP) of the improved YOLOv4 model reaches 99.26%, which is 5.9% higher than the original algorithm. Moreover, compared with Faster R-CNN, SSD and other algorithms, improved YOLOv4 has achieved excellent results. Finally, we transplanted the model to the embedded device Raspberry Pi 4B and assembled it on the UAV, using the model's lightweight and high-efficiency features to achieve flexible and fast flight detection of RIFA nests.

References

[1]	L. Lv, Y. He, J. Liu, X. Liu, S. Vinson, Invasion, spread, biology and harm of Solenopsis invicta Buren (in Chinese), Cant. Agric. Sci., 5 (2006), 3-11. https://doi.org/10.16768/j.issn.1004-874x.2006.05.001 doi: 10.16768/j.issn.1004-874x.2006.05.001
[2]	W. B. Wu, L. Zhi, T. S. Hong, et al., Detection of Solenopsis invicta nest using spectrum analysis technology, T. Chin. Soc. Agric. Eng. (Transactions of the CSAE), 29 (2013), 175-182.
[3]	R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in 2014 IEEE Conference on Computer Vision and Pattern Recognition, (2014), 580-587. https://doi.org/10.1109/CVPR.2014.81
[4]	R. Girshick, Fast R-CNN, in 2015 IEEE International Conference on Computer Vision (ICCV), (2015), 1440-1448. https://doi.org/10.1109/ICCV.2015.169
[5]	S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: towards real-time object detection with region proposal networks, in IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2016), 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
[6]	K. He, G. Gkioxari, P. Dollar, R. Girshick, Mask R-CNN, in IEEE Transactions on Pattern Analysis and Machine Intelligence, 42 (2017), 386-397. https://doi.org/10.48550/arXiv.1703.06870
[7]	J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified, real-time object detection, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 779-788. https://doi.org/10.1109/CVPR.2016.91
[8]	W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, SSD: single shot multibox detector, in European Conference on Computer Vision, (2016), 21-37. https://doi.org/10.1007/978-3-319-46448-0_2
[9]	A. Roy, J. Bhaduri, Real-time growth stage detection model for high degree of occultation using DenseNet-fused YOLOv4, Comput. Electron. Agric., 193 (2022), 106694. https://doi.org/10.1016/j.compag.2022.106694 doi: 10.1016/j.compag.2022.106694
[10]	A. Roy, R. Bose, J. Bhaduri, A fast accurate fine-grain object detection model based on YOLOv4 deep neural network, Neural Comput. Appl., 34 (2022), 3895-3921. https://doi.org/10.1007/s00521-021-06651-x doi: 10.1007/s00521-021-06651-x
[11]	M. O. Lawal, Tomato detection based on modified YOLOv3 framework, Sci. Rep., 11 (2021), 1447. https://doi.org/10.1038/s41598-021-81216-5 doi: 10.1038/s41598-021-81216-5
[12]	D. Wu, S. Lv, M. Jiang, H. Song, Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments, Comput. Electron. Agric, 178 (2020), 105742. https://doi.org/10.1016/j.compag.2020.105742 doi: 10.1016/j.compag.2020.105742
[13]	O. Agbo-Ajala, S. Viriri., A lightweight convolutional neural network for real and apparent age estimation in unconstrained face images, IEEE Access, 8 (2020), 162800-162808. https://doi.org/10.1109/ACCESS.2020.3022039 doi: 10.1109/ACCESS.2020.3022039
[14]	A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, et al., MobileNets: efficient convolutional neural networks for mobile vision applications, Comput. Sci., (2017), 1-9. https://doi.org/10.48550/arXiv.1704.04861
[15]	X. Zang, X. Zhou, M. Lin, J. Sun, ShuffleNet: an extremely efficient convolutional neural network for mobile devices, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 6848-6856. https://doi.org/10.1109/CVPR.2018.00716
[16]	M. Tan, Q. Le, EfficientNet: rethinking model scaling for convolutional neural networks, in Proceedings of the 36^th International Conference on Machine Learning, (2019), 6105-6114. https://doi.org/10.48550/arXiv.1905.11946
[17]	F. Zhang, Z. Chen, R. Bao, C. Zhang, Z. Wang, Recognition of dense cherry tomatoes based on improved YOLOv4-LITE lightweight neural network (in Chinese), Trans. Chin. Soc. Agric. Eng., 37 (2021), 270-278. https://doi.org/10.11975/j.issn.1002-6819.2021.16.033 doi: 10.11975/j.issn.1002-6819.2021.16.033
[18]	M. Togacar, B. Ergen, Classification of cloud images by using super resolution, semantic segmentation approaches and binary sailfish optimization method with deep learning model, Comput. Electron. Agric., 193 (2022), 106724. https://doi.org/10.1016/j.compag.2022.106724
[19]	X. Wang, X. Zhuang, W. Zhang, Y. Chen, Y. Li, Lightweight real-time object detection model for UAV platform, in 2021 International Conference on Computer Communication and Artificial Intelligence (CCAI), (2021), 20-24. https://doi.org/10.1109/CCAI50917.2021.9447518
[20]	L. Yu, E. Yang, C. Luo, P. Ren, AMCD: an accurate deep learning-based metallic corrosion detector for MAV-based real-time visual inspection, J. Ambient Intell. Hum. Comput., 2021 (2021), 1-12. https://doi.org/10.1007/s12652-021-03580-4 doi: 10.1007/s12652-021-03580-4
[21]	Y. Cheng, T. Zheng, Binocular visual obstacle avoidance of UAV based on deep learning (in Chinese), Elect. Opt. Control, 10 (2021), 31-35. https://doi.org/10.3969/j.issn.1671-637X.2021.10.007 doi: 10.3969/j.issn.1671-637X.2021.10.007
[22]	V. Gonzalez-Huitron, j. León-Borges, A. E. Rodriguez-Mata, L. Amabilis-Sosa, B. Ramírez-Pereda, H. Rodriguez, Disease detection in tomato leaves via CNN with lightweight architectures implemented in raspberry Pi 4, Comput. Elect. Agri., 181 (2021), 105951. https://doi.org/10.1016/j.compag.2020.105951 doi: 10.1016/j.compag.2020.105951
[23]	S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, Y. Yoo, Cutmix: regularization strategy to train strong classifiers with localizable features, in Proceedings of the IEEE International Conference on Computer Vision, (2019), 6023-6032. https://doi.org/10.1109/ICCV.2019.00612
[24]	Y. Ma, X. Cai, F. Sun, Towards no-reference image quality assessment based on multi-scale convolutional neural network, Comput. Model. Eng. Sci., 123 (2020), 201-216. https://doi.org/10.32604/cmes.2020.07867 doi: 10.32604/cmes.2020.07867
[25]	A. Bochkovskiy, C. Y. Wang, H. Y. M. Liao, YOLOv4: Optimal speed and accuracy of object detection, preprint arXiv: 10934, 2004.
[26]	Y. Wu, K. He, Group normalization, in European Conference on Computer Vision, 128 (2020), 742-755. https://doi.org/10.1007/978-3-030-01261-8_1
[27]	K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, C. Xu, GhostNet: more features from cheap operations, in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020), 1577-1586. https://doi.org/10.1109/CVPR42600.2020.00165
[28]	M. Wang, S. Jiang, J. Wu, C. Wang, Research on image defogging algorithm based on improved generative antagonistic network, J. Changchun Univ. Sci. Technol., 44 (2021), 93-99.
[29]	Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, Q. Hu, ECA-Net: efficient channel attention for deep convolutional neural networks, in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. https://doi.org/10.1109/CVPR42600.2020.01155
[30]	B. Li, Y. Q. Yao, J. Tan, G. Zhang, F. Yu, J. Lu, Equalized focal loss for dense long-tailed object detection, preprint, arXiv: 2201.02593.

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)