Driver identification and fatigue detection algorithm based on deep learning

Yuhua Ma; Ye Tao; Yuandan Gong; Wenhua Cui; Bo Wang; Yuhua Ma; Ye Tao; Yuandan Gong; Wenhua Cui; Bo Wang

doi:10.3934/mbe.2023355

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 5: 8162-8189. doi: 10.3934/mbe.2023355

Previous Article Next Article

Research article

Driver identification and fatigue detection algorithm based on deep learning

1.
School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan, China
2.
School of Electronic and Information Engineering, University of Science and Technology Liaoning, Anshan, China

Academic Editor: Yang Kuang

Received: 07 December 2022 Revised: 03 February 2023 Accepted: 12 February 2023 Published: 27 February 2023

In order to avoid traffic accidents caused by driver fatigue, smoking and talking on the phone, it is necessary to design an effective fatigue detection algorithm. Firstly, this paper studies the detection algorithms of driver fatigue at home and abroad, and analyzes the advantages and disadvantages of the existing algorithms. Secondly, a face recognition module is introduced to crop and align the acquired faces and input them into the Facenet network model for feature extraction, thus completing the identification of drivers. Thirdly, a new driver fatigue detection algorithm based on deep learning is designed based on Single Shot MultiBox Detector (SSD) algorithm, and the additional layer network structure of SSD is redesigned by using the idea of reverse residual. By adding the detection of drivers' smoking and making phone calls, adjusting the size and number of prior boxes of SSD algorithm, improving FPN network and SE network, the identification and verification of drivers can be realized. The experimental results showed that the number of parameters decreased from 96.62 MB to 18.24 MB. The average accuracy rate increased from 89.88% to 95.69%. The projected number of frames per second increased from 51.69 to 71.86. When the confidence threshold was set to 0.5, the recall rate of closed eyes increased from 46.69% to 65.87%, that of yawning increased from 59.72% to 82.72%, and that of smoking increased from 65.87% to 83.09%. These results show that the improved network model has better feature extraction ability for small targets.
- SSD algorithm,
- fatigue detection,
- facial identification,
- driving state,
- driving behavior
Citation: Yuhua Ma, Ye Tao, Yuandan Gong, Wenhua Cui, Bo Wang. Driver identification and fatigue detection algorithm based on deep learning[J]. Mathematical Biosciences and Engineering, 2023, 20(5): 8162-8189. doi: 10.3934/mbe.2023355

Related Papers:

Abstract

In order to avoid traffic accidents caused by driver fatigue, smoking and talking on the phone, it is necessary to design an effective fatigue detection algorithm. Firstly, this paper studies the detection algorithms of driver fatigue at home and abroad, and analyzes the advantages and disadvantages of the existing algorithms. Secondly, a face recognition module is introduced to crop and align the acquired faces and input them into the Facenet network model for feature extraction, thus completing the identification of drivers. Thirdly, a new driver fatigue detection algorithm based on deep learning is designed based on Single Shot MultiBox Detector (SSD) algorithm, and the additional layer network structure of SSD is redesigned by using the idea of reverse residual. By adding the detection of drivers' smoking and making phone calls, adjusting the size and number of prior boxes of SSD algorithm, improving FPN network and SE network, the identification and verification of drivers can be realized. The experimental results showed that the number of parameters decreased from 96.62 MB to 18.24 MB. The average accuracy rate increased from 89.88% to 95.69%. The projected number of frames per second increased from 51.69 to 71.86. When the confidence threshold was set to 0.5, the recall rate of closed eyes increased from 46.69% to 65.87%, that of yawning increased from 59.72% to 82.72%, and that of smoking increased from 65.87% to 83.09%. These results show that the improved network model has better feature extraction ability for small targets.

References

[1]	D. Shi, C. Sun, X. Sheng, X. Bi, Design of monitoring system for driving safety based on convolutional neural network, J. Hebei North Univ., 36 (2020), 57–61. https://doi.org/10.3969/j.issn.1673-1492.2020.09.011 doi: 10.3969/j.issn.1673-1492.2020.09.011
[2]	X. Meng, Driving fatigue caused by tramc accident characteristics and effective prevention analysis, Logist. Eng. Manage., 8 (2014), 187–188. https://doi.org/10.3969/j.issn.1674-4993.2014.08.073 doi: 10.3969/j.issn.1674-4993.2014.08.073
[3]	X. Gong, J. Fang, X. Tan, A. Liao, C. Xiao, Analysis of the current situation of road traffic accidents in the 31 provinces/municipalities of China and the projection for achieving the SDGs target of halving the numbers of death and injury, Chin. J. Dis. Control Prev., 24 (2020), 4–8. http://doi.org/10.16462/j.cnki.zhjbkz.2020.01.002 doi: 10.16462/j.cnki.zhjbkz.2020.01.002
[4]	S. Chen, J. Hu, Causative analysis of road traffic accidents and research on safety prevention measures, Leg. Syst. Soc., 27 (2020), 143–144. https://doi.org/10.19387/j.cnki.1009-0592.2020.09.247 doi: 10.19387/j.cnki.1009-0592.2020.09.247
[5]	J. Wang, X. Yu, Q. Liu, Y. Zhou, Research on key technologies of intelligent transportation based on image recognition and anti-fatigue driving, EURASIP J. Image Video Process., 1 (2019), 33–45. https://doi.org/10.1186/s13640-018-0403-6 doi: 10.1186/s13640-018-0403-6
[6]	X. Wang, R. Chen, B. Huang, Implementation of driver driving safety monitoring system based on android system, Electron. Meas. Technol., 42 (2019), 56–60. https://doi.org/10.19651/j.cnki.emt.1802406 doi: 10.19651/j.cnki.emt.1802406
[7]	S. Liu, L. He, Fatigue driving detection system based on image processing, J. Yuncheng Univ., 39 (2021), 51–54. https://doi.org/10.15967/j.cnki.cn14-1316/g4.2021.06.013 doi: 10.15967/j.cnki.cn14-1316/g4.2021.06.013
[8]	F. Liu, D. Chen, J. Zhou, F. Xu, A review of driver fatigue detection and its advances on the use of RGB-D camera and deep learning, Eng. Appl. Artif. Intell., 116 (2022), 105399. https://doi.org/10.1016/j.engappai.2022.105399 doi: 10.1016/j.engappai.2022.105399
[9]	Y. Sui, Z. Yan, L. Dai. H. Jing, Face multi-attribute detection algorithm based on RetinaFace, Railway Comput. Appl., 30 (2021), 1–4. https://doi.org/10.3969/j.issn.1005-8451.2021.03.001 doi: 10.3969/j.issn.1005-8451.2021.03.001
[10]	S. Yang, P. Luo, C. C. Loy, X. Tang, Wider face: a face detection benchmark, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 5525–5533. https://doi.org/10.1109/CVPR.2016.596
[11]	G. M. Clayton, S. Devasia, Image-based compensation of dynamic effects in scanning tunnelling microscopes, Nanotechnology, 16 (2005), 809–818. https://doi.org/10.1088/0957-4484/16/6/032 doi: 10.1088/0957-4484/16/6/032
[12]	L. Huang, H. Yang, B. Wang, Research and improvement of multi-method combined face image illumination compensation algorithm, J. Chongqing Univ. Technol., 31 (2017), 6–12. https://doi.org/10.3969/j.issn.1674-8425(z).2017.11.027 doi: 10.3969/j.issn.1674-8425(z).2017.11.027
[13]	L. Shao, R. Yan, X. Li, Y. Liu, From heuristic optimization to dictionary learning: A review and comprehensive comparison of Image denoising algorithms, IEEE Trans. Cybern., 44 (2017), 1001–1013. https://doi.org/10.1109/TCYB.2013.2278548 doi: 10.1109/TCYB.2013.2278548
[14]	C. Shi, C. Zhang, Q. He, H. Wang, Target detection based on improved feature pyramid, Electron. Meas. Technol., 44 (2021), 150–156. https://doi.org/10.19651/j.cnki.emt.2107598 doi: 10.19651/j.cnki.emt.2107598
[15]	X. Guo, Research on Multi-Scale Face Detection Based on Convolution Neural Networks, M.S thesis, North China Electric Power University in Hebei, 2020.
[16]	F. Chen, Research on Cosine Loss Algorithm for Face Verification, M.S thesis, Xiangtan University in Hunan, 2020. https://doi.org/10.27426/d.cnki.gxtdu.2020.001269
[17]	Z. Yang, L. Hou, D. Yang, lmproved face recognition algorithm of attitude correction, Cyber Secur. Data Governance, 35 (2016), 56–60. https://doi.org/10.19358/j.issn.1674-7720.2016.03.019 doi: 10.19358/j.issn.1674-7720.2016.03.019
[18]	S. Preetha, S. V. Sheela, Security monitoring system using facenet for wireless sensor network, preprint, arXiv: 2112.01305.
[19]	X. Li, R. Huang, Z. Chen, Y. Long, L. Xu, An improved face detection and recognition algorithm based on FaceNet and MTCNN, J. Guangdong Univ. of Petrochem. Technol., 31 (2021), 45–47.
[20]	J. Wang, J. Li, X. Zhou, X. Zhang, Improved SSD algorithm and its performance analysis of small target detection in remote sensing images, Acta Opt. Sin. 39 (2019), 10. https://doi.org/10.3788/AOS201939.0628005 doi: 10.3788/AOS201939.0628005
[21]	S. Mao, H. Li, Research on improved SSD algorithm for detection in traffic, Microprocessors, 43 (2022), 26–29.
[22]	B. Wang, Y. Lv, X. Hei, H. Jin, Lightweight deep convolutional neural network model based on dilated convolution, 2020. Available from: https://kns.cnki.net/kcms2/article/abstract?v = kxaUMs6x7-4I2jr5WTdXti3zQ9F92xu0dKxhnJcY9pxwfrkG2rAGFOJWdZMiOIJjZJ9FLVWmYcCCgfpgeyHSjqedCLDh_ut5 & uniplatform = NZKPT
[23]	L. Jiang, J. Li, B. Huang, Research on face feature detection algorithm based on improved SSD, Mach. Des. Manuf. Eng., 50 (2021), 82–86. https://doi.org/10.3969/j.issn.2095-509X.2021.07.017 doi: 10.3969/j.issn.2095-509X.2021.07.017
[24]	X. Zhang, A. Jiang, SSD Small Target detection algorithm combining feature enhancement and self-attention, Comput. Eng. Appl., 58 (2022), 247–255. https://doi.org/10.3778/j.issn.1002-8331.2109-0356 doi: 10.3778/j.issn.1002-8331.2109-0356
[25]	J. Guo, T. Yu, Y. Cui, X. Zhou, Research on vehicle small target detection algorithm based on improved SSD, Comput. Technol. Dev., 32 (2022), 1–7.
[26]	Q. Zheng, L. Wang, F. Wang, Candidate box generation method based on improved ssd network, 2020. Available from: https://kns.cnki.net/kcms2/article/abstract?v = kxaUMs6x7-4I2jr5WTdXti3zQ9F92xu0ManZHCyoNk-lwS3y-OLIR4fcD18PUKrUkLhyHScAkvpkTgimuL-OfVjGi7Jisy2h & uniplatform = NZKPT
[27]	Q. Song, X. Wang, C. Zhang, Y. Chen, H. Song, A residual SSD model based on window size clustering for traffic sign detection, J. Hunan Univ., 46 (2019), 133–140. https://doi.org/10.16339/j.cnki.hdxbzkb.2019.10.016 doi: 10.16339/j.cnki.hdxbzkb.2019.10.016
[28]	W. Chen, Lightweight convolutional neural network remote sensing image target detection, Beijing Surv. Mapp., 36 (2018), 178–183. https://doi.org/10.19580/j.cnki.1007-3000.2022.02.014 doi: 10.19580/j.cnki.1007-3000.2022.02.014
[29]	K. Chen, Research on SSD-based Multi-scale Detection Algorithm, M.S thesis, Beijing Jiaotong University in Beijing, 2020. https://doi.org/10.26944/d.cnki.gbfju.2020.002225
[30]	H. Zhang, M. Zhang, SSD Target Detection Algorithm with Channel Attention Mechanism, Comput. Eng., 46 (2020), 264–270. https://doi.org/10.19678/j.issn.1000-3428.0054946 doi: 10.19678/j.issn.1000-3428.0054946
[31]	Z. A. Haq, Z. Hasan, Eye-blink rate detection for fatigue determination, in 2016 1st India International Conference on Information Processing (IICIP), (2016), 1–5. https://doi.org/10.1109/IICIP.2016.7975348
[32]	X. Zhou, S. Wang, W. Zhao, X. Zhao, T. Li, Fatigue Driving Detection Based on State Recognition of Eyes and Mouth, J. Jilin Univ., 35 (2017), 204–211. https://doi.org/10.19292/j.cnki.jdxxp.2017.02.015 doi: 10.19292/j.cnki.jdxxp.2017.02.015

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)