Convolutional neural network based obstacle detection for unmanned surface vehicle

Liyong Ma; Wei Xie; Haibin Huang; Liyong Ma; Wei Xie; Haibin Huang

doi:10.3934/mbe.2020045

Mathematical Biosciences and Engineering

2020, Volume 17, Issue 1: 845-861. doi: 10.3934/mbe.2020045

Previous Article Next Article

Research article Special Issues

Convolutional neural network based obstacle detection for unmanned surface vehicle

School of Information Science and Engineering, Harbin Institute of Technology, Weihai 264209, China

Received: 30 April 2019 Accepted: 22 October 2019 Published: 05 November 2019

Unmanned surface vehicles (USV) is the development trend of future ships, and it will be widely used in various kinds of marine tasks. Obstacle avoidance is one key technology for autonomous navigation of USV. Convolutional neural network based obstacle classification and detection method is applied to USV visual images in environment sensing task. To solve the problem of low detection and classification accuracy of obstacles in the visual inspection of USV, a bidirectional feature pyramid networks is proposed combining hybrid network architecture of ResNet and improved DenseNet. The proposed method can further enhance the detection and classification some types of obstacles by using the underlying multi-layer detail features and high-level strong semantic features in the network architecture. The detection and classification performance of the proposed method is evaluated on a self built dataset. Ablation experiments and performance tests on open datasets are also employed. The experimental results show that the proposed algorithm has best performance for obstacles detection, and it is more suitable for autonomous navigation of USV.
- unmanned surface vehicle,
- convolutional neural network,
- obstacle detection,
- deep learning,
- feature pyramid network
Citation: Liyong Ma, Wei Xie, Haibin Huang. Convolutional neural network based obstacle detection for unmanned surface vehicle[J]. Mathematical Biosciences and Engineering, 2020, 17(1): 845-861. doi: 10.3934/mbe.2020045

Related Papers:

Abstract

Unmanned surface vehicles (USV) is the development trend of future ships, and it will be widely used in various kinds of marine tasks. Obstacle avoidance is one key technology for autonomous navigation of USV. Convolutional neural network based obstacle classification and detection method is applied to USV visual images in environment sensing task. To solve the problem of low detection and classification accuracy of obstacles in the visual inspection of USV, a bidirectional feature pyramid networks is proposed combining hybrid network architecture of ResNet and improved DenseNet. The proposed method can further enhance the detection and classification some types of obstacles by using the underlying multi-layer detail features and high-level strong semantic features in the network architecture. The detection and classification performance of the proposed method is evaluated on a self built dataset. Ablation experiments and performance tests on open datasets are also employed. The experimental results show that the proposed algorithm has best performance for obstacles detection, and it is more suitable for autonomous navigation of USV.

References

[1]	M. Schiaretti, L. Chen and R. Negenborn, Survey on autonomous surface vessels: Part IA new detailed definition of autonomy levels, International Conference on Computational Logistics, 2017, 219-233. Available from: https://link_springer.xilesou.top/chapter/10.1007/978-3-319-68496-3_15.
[2]	D. NaÄŚ, N. MiÅąkoviÄǦ and F. MandiÄǦ, Navigation, guidance and control of an overactuated marine surface vehicle, Annu. Rev. Control, 40 (2015), 172-181.
[3]	M. Schuster, M. Blaich and J. Reuter, Collision avoidance for vessels using a low-cost radar sensor IFAC Proc. Vol., 2014 (2014), 9673-9678.
[4]	S. Kim and J. Lee, Small infrared target detection by region-adaptive clutter rejection for sea-based infrared search and track, Sensors, 14 (2014), 13210-13242.
[5]	H. Wang, X. Mou, W. Mou, et al., Vision based long range object detection and tracking for unmanned surface vehicle, 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), 2015, 101-105. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/7274604/.
[6]	Y. Liu, L. Ma, W. Xie, et al., Parallel GPU computation model for block matching of speckle tracing, J. Nonlinear Convex Anal., 20 (2019), 827-833.
[7]	D. Prasad, C. Prasath, D. Rajan, et al., Object detection in a maritime environment: Performance evaluation of background subtraction methods, IEEE Trans. Intell. Transp. Syst., 20 (2019), 1787-1802.
[8]	C. Osborne, T. Cane, T. Nawaz, et al., Temporally stable feature clusters for maritime object tracking in visible and thermal imagery, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2015, 1-6. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/7301769.
[9]	T. Cane and J. Ferryman, Saliency-based detection for maritime object tracking, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016, 18-25. Available from: https://www.cvfoundation.org/openaccess/content_cvpr_2016_workshops/w20/html/Cane_SaliencyBased_Detection_for_CVPR_2016_paper.html.
[10]	M. Kristan, V. Kenk, S. KovaÄDiÄD, et al., Fast image-based obstacle detection from unmanned surface vehicles, IEEE Trans. Cybern., 46 (2016), 641-654.
[11]	B. Bovcon and M. Kristan, Obstacle detection for USVs by joint stereo-view semantic segmentation, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, 5807-5812. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/8594238.
[12]	D. K. Prasad, D. Rajan, L. Rachmawati, et al., Video processing from electro-optical sensors for object detection and tracking in a maritime environment: A survey, IEEE Trans. Intell. Transp. Syst., 18 (2017), 1993-2016.
[13]	S. Ren, K. He, R. Girshick, et al., Faster R-CNN: Towards real-time object detection with region proposal networks, Advances in neural information processing systems, 2017, 1137-1149. Available from: http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detectionwith-region-proposal-networks.
[14]	J. Redmon and F. Ali, YOLO9000: Better, faster, stronger, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 6517-6525. Available from: http://openaccess.thecvf.com/content_cvpr_2017/html/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.html.
[15]	K. He, G. Gkioxari, P. Dollar, et al., Mask R-CNN, The IEEE International Conference on Computer Vision (ICCV), 2017, 2961-2969. Available from: http://openaccess.thecvf.com/content_iccv_2017/html/He_Mask_R-CNN_ICCV_2017_paper.html.
[16]	S. Pang, J. Coz, Z. Yu, et al., Deep learning to frame objects for visual target tracking, Eng. Appl. Artif. Intell., 65 (2017), 406-420.
[17]	Y. Long, Y. Gong, Z. Xiao, et al., Accurate object localization in remote sensing images based on convolutional neural networks, IEEE Trans. Geosci. Remote Sens., 55 (2017), 2486-2498.
[18]	Y. LeCun, Y. Bengio and G. Hinton, Deep learning, Nature, 512 (2015), 336-444.
[19]	Y. Chen, J. Li and H. Xiao, et al, Dual path network, Advanced in Neural Information Processing Systems, 2017, 4468-4476. Available from: http://papers.nips.cc/paper/7033-dual-path-networks.
[20]	A. Kumar and E. Sherly, A convolutional neural network for visual object recognition in marine sector, 2017 2nd International Conference for Convergence in Technology (I2CT), 2017, 304-307. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/8226141.
[21]	J. Yang, Y. Li, Q. Zhang, et al., Surface vehicle detection and tracking with deep learning and appearance feature, 2019 5th International Conference on Control, Automation and Robotics (ICCAR), 2019, 276-280. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/8813345.
[22]	T. Cane and J. Ferryman, Evaluating deep semantic segmentation networks for object detection in maritime surveillance, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2018, 1-6. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/8639077.
[23]	H. Fu, Y. Li, Y. Wang, et al., Maritime target detection method based on deep learning, 2018 IEEE International Conference on Mechatronics and Automation (ICMA), 2018, 878-883. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/8484727.
[24]	L. Qu, S. Wang, N. Yang, et al., Improving object detection accuracy with region and regression based deep CNNs, 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), 2017, 318-323. Available from: https://ieeexplore_ieee.xilesou.top/abstract/document/8304297.
[25]	W. Chen, J. Li, J. Xing, et al., A maritime targets detection method based on hierarchical and multi-scale deep convolutional neural network, Tenth International Conference on Digital Image Processing (ICDIP 2018). International Society for Optics and Photonics, 2018, 1080616. Available from: https://www.spiedigitallibrary.org/conference-proceedingsof-spie/10806/1080616/A-maritime-targets-detection-method-based-on-hierarchical-andmulti/10.1117/12.2503030.short?SSO=1.
[26]	S. Jia, L. Ma and S. Zhang, Big data prototype practice for unmanned surface vehicle, ICCIP'18 Proceedings of the 4th International Conference on Communication and Information Processing, 2018, 43-47. Available from: https://dl_acm.xilesou.top/citation.cfm?id=3290466.
[27]	L. Y. Ma, C. K. Ma, Y. J. Liu, et al., Thyroid diagnosis from SPECT images using convolutional neural network with optimization, Comput. Intell. Neurosci., 2019 (2019), 6212759.
[28]	L. Y. Ma, W. Xie and Y. Zhang, Blister defect detection based on convolutional neural network for polymer lithium-ion battery, Appl. Sci., 9 (2019), 1085.
[29]	S. Pouyanfar, S. Sadiq, Y. Yan, et al., A survey on deep learning: Algorithms, techniques, and applications, ACM Comput. Surv., 51 (2019), 92.
[30]	W. Rawat and Z. Wang, Deep convolutional neural networks for image classification: A comprehensive review, Neural Comput., 29 (2017), 2352-2449.
[31]	K. He, X. Zhang, S. Ren, et al., Deep residual learning for image recognition, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 770-778. Available from: http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html.
[32]	G. Huang, Z. Liu, L. Van Der Maater, et al., Densely connected convolutional networks, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 4700-4708. Available from: http://openaccess.thecvf.com/content_cvpr_2017/html/Huang_Densely_Connected_Convolutional_CVPR_2017_paper.html.
[33]	W. Liu, D. Auguelov, D. Erhan, et al., SSD: Single shot multibox detector, European Conference on Computer Vision, 2016, 21-37. Available from: https://link_springer.xilesou.top/chapter/10.1007/978-3-319-46448-0_2.
[34]	T-Y. Lin, P. Dollar, R. Girshick, et al, Feature pyramid network for object detection, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 2117-2125. Available from: http://openaccess.thecvf.com/content_cvpr_2017/html/Lin_Feature_Pyramid_Networks_CVPR_2017_paper.html.

Reader Comments

Your name:*

Email:*
© 2020 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)