Effect of Gaussian filtered images on Mask RCNN in detection and segmentation of potholes in smart cities

Auwalu Saleh Mubarak; Zubaida Said Ameen; Fadi Al-Turjman; Auwalu Saleh Mubarak; Zubaida Said Ameen; Fadi Al-Turjman

doi:10.3934/mbe.2023013

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 1: 283-295. doi: 10.3934/mbe.2023013

Previous Article Next Article

Research article Special Issues

Effect of Gaussian filtered images on Mask RCNN in detection and segmentation of potholes in smart cities

1.
Artificial Intelligence Engineering Dept., AI and Robotics Institute, Near East University, Mersin 10, Turkey
2.
Electrical Engineering Dept., Kano University of Science and Technology, Wudil, Kano, Nigeria
3.
Biochemistry Dept., Maitama Sule University, Kano, Nigeria
4.
Research Center for AI and IoT, Faculty of Engineering, University of Kyrenia, Kyrenia, Mersin 10, Turkey

Academic Editor: Byung-Gyu Kim

Received: 26 July 2022 Revised: 30 August 2022 Accepted: 09 September 2022 Published: 30 September 2022

Accidents have contributed a lot to the loss of lives of motorists and serious damage to vehicles around the globe. Potholes are the major cause of these accidents. It is very important to build a model that will help in recognizing these potholes on vehicles. Several object detection models based on deep learning and computer vision were developed to detect these potholes. It is very important to develop a lightweight model with high accuracy and detection speed. In this study, we employed a Mask RCNN model with ResNet-50 and MobileNetv1 as the backbone to improve detection, and also compared the performance of the proposed Mask RCNN based on original training images and the images that were filtered using a Gaussian smoothing filter. It was observed that the ResNet trained on Gaussian filtered images outperformed all the employed models.
- Mask RCNN,
- pothole,
- computer vision,
- smart cities,
- object detection,
- Gaussian filter
Citation: Auwalu Saleh Mubarak, Zubaida Said Ameen, Fadi Al-Turjman. Effect of Gaussian filtered images on Mask RCNN in detection and segmentation of potholes in smart cities[J]. Mathematical Biosciences and Engineering, 2023, 20(1): 283-295. doi: 10.3934/mbe.2023013

Related Papers:

Abstract

Accidents have contributed a lot to the loss of lives of motorists and serious damage to vehicles around the globe. Potholes are the major cause of these accidents. It is very important to build a model that will help in recognizing these potholes on vehicles. Several object detection models based on deep learning and computer vision were developed to detect these potholes. It is very important to develop a lightweight model with high accuracy and detection speed. In this study, we employed a Mask RCNN model with ResNet-50 and MobileNetv1 as the backbone to improve detection, and also compared the performance of the proposed Mask RCNN based on original training images and the images that were filtered using a Gaussian smoothing filter. It was observed that the ResNet trained on Gaussian filtered images outperformed all the employed models.

References

[1]	V. L. Solanke, D. D. Patil, A. S. Patkar, G. S. Tamrale, A. G. Kale, Analysis of existing road surface on the basis of pothole characteristics, Global J. Res. Eng., 19 (2019).
[2]	City of San Antonio 311 City Services and Info, Potehole/pavement reapair, 2018. Available from: https://311.sanantonio.gov/kb/docs/articles/transportation/potholes
[3]	V. Pandey, K. Anand, A. Kalra, A. Gupta, P. P. Roy, B. G. Kim, Enhancing object detection in aerial images, Math. Biosci. Eng., 19 (2022), 7920–7932, https://doi.org/10.3934/mbe.2022370 doi: 10.3934/mbe.2022370
[4]	S. M. Hejazi, C. Abhayaratne, Handcrafted localized phase features for human action recognition, Image Vis. Comput., 123 (2022), 104465. https://doi.org/10.1016/j.imavis.2022.104465 doi: 10.1016/j.imavis.2022.104465
[5]	A. A. Mohamed, F. Alqahtani, A. Shalaby, A. Tolba, Texture classification-based feature processing for violence-based anomaly detection in crowded environments, Image Vis. Comput., 124 (2022), 104465. https://doi.org/10.1016/j.imavis.2022.104488 doi: 10.1016/j.imavis.2022.104488
[6]	Z. Qu, L. Y. Gao, S. Y. Wang, H. N. Yin, T. M. Yi, An improved YOLOv5 method for large objects detection with multi-scale feature cross-layer fusion network, Image Vis. Comput., 125 (2022), 104518. https://doi.org/10.1016/j.imavis.2022.104518 doi: 10.1016/j.imavis.2022.104518
[7]	K. He, G. Gkioxari, P. Dollar, R. Girshick, Mask R-CNN, in 2017 IEEE International Conference on Computer Vision (ICCV), IEEE, Venice, Italy, (2020), 2980–2988. https://doi.org/10.1109/ICCV.2017.322
[8]	A. S. Mubarak, S. Serte, F. Al‐Turjman, Z. S. Ameen, M. Ozsoz, Local binary pattern and deep learning feature extraction fusion for COVID‐19 detection on computed tomography images, Expert Syst., 39 (2022), e12842. https://doi.org/10.1111/exsy.12842 doi: 10.1111/exsy.12842
[9]	M. Ozsoz, A. Mubarak, Z. Said, R. Aliyu, F. Al Turjman, S. Serte, Deep learning-based feature extraction coupled with multi-class SVM for COVID-19 detection in the IoT era, Int. J. Nanotechnol., 1 (2021). https://doi.org/10.1504/ijnt.2021.10040115 doi: 10.1504/ijnt.2021.10040115
[10]	J. Redmon, A. Farhadi, YOLOv3: An incremental improvement, preprint, arXiv: 1804.02767.
[11]	G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Honolulu, USA, (2017), 2261–2269. https://doi.org/10.1109/CVPR.2017.243
[12]	B. X. Yu, X. Yu, Vibration-based system for pavement condition evaluation, in Ninth International Conference on Applications of Advanced Technology in Transportation, (2006), 183–189. https://doi.org/10.1061/40799(213)31
[13]	K. De Zoysa, C. Keppitiyagama, G. P. Seneviratne, W. W. A. T. Shihan, A public transport system based sensor network for road surface condition monitoring, in Proceedings of the 2007 workshop on Networked systems for developing regions, ACM, Kyoto, Japan, (2007), 1–6. https://doi.org/10.1145/1326571.1326585
[14]	M. B. Sai Ganesh Naik, V. Nirmalrani, Detecting potholes using image processing techniques and real-world footage, in Cognitive Informatics and Soft Computing, Springer, (2021), 893–902. https://doi.org/10.1007/978-981-16-1056-1_72
[15]	L. Huidrom, L. K. Das, S. K. Sud, Method for automated assessment of potholes, cracks and patches from road surface video clips, Procedia-Soc. Behav. Sci., 104 (2013), 312–321. https://doi.org/10.1016/j.sbspro.2013.11.124 doi: 10.1016/j.sbspro.2013.11.124
[16]	J. Lin, Y. Liu, Potholes detection based on SVM in the pavement distress image, in 2010 Ninth International Symposium on Distributed Computing and Applications to Business, Engineering and Science, IEEE, Hong Kong, China, (2010), 544–547. https://doi.org/10.1109/DCABES.2010.115
[17]	M. H. Yousaf, K. Azhar, F. Murtaza, F. Hussain, Visual analysis of asphalt pavement for detection and localization of potholes, Adv. Eng. Inf., 38 (2018), 527–537. https://doi.org/10.1016/j.aei.2018.09.002 doi: 10.1016/j.aei.2018.09.002
[18]	A. Dhiman, R. Klette, Pothole detection using computer vision and learning, IEEE Trans. Intell. Transp. Syst., 21 (2020), 3536–3550. https://doi.org/10.1109/TITS.2019.2931297 doi: 10.1109/TITS.2019.2931297
[19]	S. K. Sharma, S. Mohapatra, R. C. Sharma, S. Alturjman, C. Altrjman, L. Mostarda, et al., Retrofitting existing buildings to improve energy performance, Sustainability, 14 (2022), 666. https://doi.org/10.3390/su14020666 doi: 10.3390/su14020666
[20]	A. S. Mubarak, Z. S. Ameen, P. Tonga, C. Altrjman, F. Al-Turjman, A framework for pothole detection via the AI-Blockchain integration, in Lecture Notes on Data Engineering and Communications Technologies, Springer, (2022), 398–406. https://doi.org/10.1007/978-3-030-99616-1_53
[21]	J. Eriksson, L. Girod, B. Hull, R. Newton, S. Madden, H. Balakrishnan, The pothole patrol: Using a mobile sensor network for road surface monitoring, in Proceedings of the 6th International Conference on Mobile Systems, ACM, Breckenridge, USA, (2008), 29–39. https://doi.org/10.1145/1378600.1378605
[22]	A. Mednis, G. Strazdins, R. Zviedris, G. Kanonirs, L. Selavo, Real time pothole detection using Android smartphones with accelerometers, in 2011 International Conference on Distributed Computing in Sensor Systems and Workshops (DCOSS), IEEE, Barcelona, Spain, (2011), 1–6. https://doi.org/10.1109/DCOSS.2011.5982206
[23]	X. Yu, E. Salari, Pavement pothole detection and severity measurement using laser imaging, in 2011 IEEE International Conference On Electro/Information Technology, IEEE, Mankato, USA, (2011), 1–5. https://doi.org/10.1109/EIT.2011.5978573
[24]	I. Moazzam, K. Kamal, S. Mathavan, S. Usman, M. Rahman, Metrology and visualization of potholes using the microsoft kinect sensor, in 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), IEEE, The Hague, Netherlands, (2013), 1284–1291. https://doi.org/10.1109/ITSC.2013.6728408
[25]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Las Vegas, USA, (2016), 770–778. https://doi.org/10.1109/CVPR.2016.90
[26]	C. T. Hendrickson, Applications of advanced technologies in transportation engineering, J. Transp. Eng., 130 (2004), 272–273. https://doi.org/10.1061/(ASCE)0733-947X(2004)130:3(272) doi: 10.1061/(ASCE)0733-947X(2004)130:3(272)
[27]	C. Koch, I. Brilakis, Pothole detection in asphalt pavement images, Adv. Eng. Inf., 25 (2011), 507–515. https://doi.org/10.1016/j.aei.2011.01.002 doi: 10.1016/j.aei.2011.01.002
[28]	M. B. Sai Ganesh Naik, V. Nirmalrani, Detecting potholes using image processing techniques and real-world footage, 1317 (2021), 893–902. https://doi.org/10.1007/978-981-16-1056-1_72
[29]	Z. Zhang, X. Ai, C. K. Chan, N. Dahnoun, An efficient algorithm for pothole detection using stereo vision, in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Florence, Italy, (2014), 564–568. https://doi.org/10.1109/ICASSP.2014.6853659
[30]	M. Saleh, Z. S. Ameen, C. Altrjman, F. Al-turjman, Computer-vision-based statue detection with gaussian smoothing filter and efficientdet, Sustainability, 14 (2022), 11413. https://doi.org/10.3390/su141811413 doi: 10.3390/su141811413
[31]	T. Chen, L. Lin, X. Wu, N. Xiao, X. Luo, Learning to segment object candidates via recursive neural networks, IEEE Trans. Image Process., 27 (2018), 5827–5839. https://doi.org/10.1109/TIP.2018.2859025 doi: 10.1109/TIP.2018.2859025
[32]	Y. Li, H. Qi, J. Dai, X. Ji, Y. Wei, Fully convolutional instance-aware semantic segmentation, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Honolulu, USA, (2017), 4438–4446, . https://doi.org/10.1109/CVPR.2017.472
[33]	X. Rong, C. Yi, Y. Tian, Unambiguous scene text segmentation with referring expression comprehension, IEEE Trans. Image Process., 29 (2020), 591–601. https://doi.org/10.1109/TIP.2019.2930176 doi: 10.1109/TIP.2019.2930176
[34]	Y. Qiao, M. Truman, S. Sukkarieh, Cattle segmentation and contour extraction based on Mask R-CNN for precision livestock farming, Comput. Electron. Agric., 165 (2019), 104958. https://doi.org/10.1016/j.compag.2019.104958 doi: 10.1016/j.compag.2019.104958
[35]	X. Liu, D. Zhao, W. Jia, W. Ji, C. Ruan, Y. Sun, Cucumber fruits detection in greenhouses based on instance segmentation, IEEE Access, 7 (2019), 139635–139642. https://doi.org/10.1109/ACCESS.2019.2942144 doi: 10.1109/ACCESS.2019.2942144
[36]	R. Sagues-Tanco, L. Benages-Pardo, G. Lopez-Nicolas, S. Llorente, Fast synthetic dataset for kitchen object segmentation in deep learning, IEEE Access, 8 (2020), 220496–220506. https://doi.org/10.1109/ACCESS.2020.3043256 doi: 10.1109/ACCESS.2020.3043256
[37]	A. M. M. Sizkouhi, M. Aghaei, S. M. Esmailifar, M. R. Mohammadi, F. Grimaccia, Automatic boundary extraction of large-scale photovoltaic plants using a fully convolutional network on aerial imagery, IEEE J. Photovoltaics, 10 (2020), 1061–1067. https://doi.org/10.1109/JPHOTOV.2020.2992339 doi: 10.1109/JPHOTOV.2020.2992339
[38]	Q. Zhang, X. Chang, S. B. Bian, Vehicle-damage-detection segmentation algorithm based on improved mask RCNN, IEEE Access, 8 (2020), 6997–7004. https://doi.org/10.1109/ACCESS.2020.2964055 doi: 10.1109/ACCESS.2020.2964055
[39]	T. DeVries, G. W. Taylor, Improved regularization of convolutional neural networks with cutout, preprint, arXiv: 1708.04552.
[40]	F. Song, L. Wu, G. Zheng, X. He, G. Wu, Y. Zhong, Multisize plate detection algorithm based on improved Mask RCNN, in 2020 IEEE International Conference on Smart Internet of Things (SmartIoT), IEEE, Beijing, China, (2020), 277–281. https://doi.org/10.1109/SmartIoT49966.2020.00049
[41]	T. Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, S. Belongie, Feature pyramid networks for object detection, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Honolulu, USA, (2017), 936–944. https://doi.org/10.1109/CVPR.2017.106
[42]	L. T. Bienias, J. R. Guillamón, L. H. Nielsen, T. S. Alstrøm, Insights into the behaviour of multi-task deep neural networks for medical image segmentation, in 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), IEEE, Pittsburgh, USA, (2019), 1–6. https://doi.org/10.1109/MLSP.2019.8918753
[43]	E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), 640–651. https://doi.org/10.1109/TPAMI.2016.2572683 doi: 10.1109/TPAMI.2016.2572683
[44]	A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, et al., Mobilenets: Efficient convolutional neural networks for mobile vision applications, preprint, arXiv: 1704.04861.
[45]	W. Wang, Y. Li, T. Zou, X. Wang, J. You, Y. Luo, A novel image classification approach via dense-MobileNet models, Mobile Inf. Syst., 2020 (2020), 7602384. https://doi.org/10.1155/2020/7602384 doi: 10.1155/2020/7602384
[46]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Vegas, USA, (2016), 770–778. https://doi.org/10.1109/CVPR.2016.90
[47]	M. Wang, S. Zheng, X. Li, X. Qin, A new image denoising method based on Gaussian filter, in 2014 International Conference on Information Science, Electronics and Electrical Engineering, IEEE, Sapporo, Japan, 1 (2014), 163–167. https://doi.org/10.1109/InfoSEEE.2014.6948089
[48]	A. R. Chitholian, Pothole Dataset, 2020. Available from: https://www.kaggle.com/datasets/chitholian/annotated-potholes-dataset.
[49]	A. Dutta, A. Zisserman, The VIA annotation software for images, audio and video, in Proceedings of the 27th ACM International Conference on Multimedia, ACM, Nice, France, (2019), 2276–2279. https://doi.org/10.1145/3343031.3350535

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)