DeepFireNet: A real-time video fire detection method based on multi-feature fusion

Bin Zhang; Linkun Sun; Yingjie Song; Weiping Shao; Yan Guo; Fang Yuan; Bin Zhang; Linkun Sun; Yingjie Song; Weiping Shao; Yan Guo; Fang Yuan

doi:10.3934/mbe.2020397

Mathematical Biosciences and Engineering

2020, Volume 17, Issue 6: 7804-7818. doi: 10.3934/mbe.2020397

Previous Article Next Article

Research article

DeepFireNet: A real-time video fire detection method based on multi-feature fusion

1.
School of Computer Science and Technology, Shandong Technology and Business University, Yantai, Shandong 264005, China
2.
Fire Rescue Brigade, Yantai Hi-tech Industry Development Zone, Yantai, Shandong 264670, China
3.
Plaza Park Management Service Center, Yantai Laishan Zone, Yantai, Shandong 264670, China

Received: 04 August 2020 Accepted: 25 October 2020 Published: 09 November 2020

This paper proposes a real-time fire detection framework DeepFireNet that combines fire features and convolutional neural networks, which can be used to detect real-time video collected by monitoring equipment. DeepFireNet takes surveillance device video stream as input. To begin with, based on the static and dynamic characteristics of fire, a large number of non-fire images in the video stream are filtered. In the process, for the fire images in the video stream, the suspected fire area in the image is extracted. Eliminate the influence of light sources, candles and other interference sources to reduce the interference of complex environments on fire detection. Then, the algorithm encodes the extracted region and inputs it into DeepFireNet convolution network, which extracts the depth feature of the image and finally judges whether there is a fire in the image. DeepFireNet network replaces 5×5 convolution kernels in the inception layer with two 3×3 convolution kernels, and only uses three improved inception layers as the core architecture of the network, which effectively reduces the network parameters and significantly reduces the amount of computation. The experimental results show that this method can be applied to many different indoor and outdoor scenes. Besides, the algorithm effectively meets the requirements for the accuracy and real-time of the detection algorithm in the process of real-time video detection. This method has good practicability.
- fire recognition,
- feature fusion,
- convolutional neural network,
- real-time video
Citation: Bin Zhang, Linkun Sun, Yingjie Song, Weiping Shao, Yan Guo, Fang Yuan. DeepFireNet: A real-time video fire detection method based on multi-feature fusion[J]. Mathematical Biosciences and Engineering, 2020, 17(6): 7804-7818. doi: 10.3934/mbe.2020397

Related Papers:

Abstract

This paper proposes a real-time fire detection framework DeepFireNet that combines fire features and convolutional neural networks, which can be used to detect real-time video collected by monitoring equipment. DeepFireNet takes surveillance device video stream as input. To begin with, based on the static and dynamic characteristics of fire, a large number of non-fire images in the video stream are filtered. In the process, for the fire images in the video stream, the suspected fire area in the image is extracted. Eliminate the influence of light sources, candles and other interference sources to reduce the interference of complex environments on fire detection. Then, the algorithm encodes the extracted region and inputs it into DeepFireNet convolution network, which extracts the depth feature of the image and finally judges whether there is a fire in the image. DeepFireNet network replaces 5×5 convolution kernels in the inception layer with two 3×3 convolution kernels, and only uses three improved inception layers as the core architecture of the network, which effectively reduces the network parameters and significantly reduces the amount of computation. The experimental results show that this method can be applied to many different indoor and outdoor scenes. Besides, the algorithm effectively meets the requirements for the accuracy and real-time of the detection algorithm in the process of real-time video detection. This method has good practicability.

References

[1]	B. U. Toreyin, Y. Dedeoglu, A. E. Cetin, Flame detection in video using hidden Markov models, IEEE International Conference on Image Processing, 2 (2005), II-1230.
[2]	O. Gunay, K. Taşdemir, B. U. Toreyin, A. E. Çetin, Fire detection in video using LMS based active learning, Fire Technol., 46 (2010), 551-577.
[3]	H. Zhao, S. Zuo, M. Hou, W. Liu, L. Yu, X. Yang, et al., A novel adaptive signal processing method based on enhanced empirical wavelet transform technology, Sensors, 18 (2018), 3323. doi: 10.3390/s18103323
[4]	J. Y. Sun, S. Y. Qi, Design in Fire Prevention Based on Multi-Sensor and WSN, Appl. Mech. Mater., 713 (2015), 2237-2240.
[5]	J. Sun, H. Jin, Intelligent design in fire prevention based on WSN, 2011 International Conference on Uncertainty Reasoning and Knowledge Engineering. IEEE, 2 (2011), 169-172.
[6]	W. Deng, J. Xu, Y. Song, H. Zhao, Differential evolution algorithm with wavelet basis function and optimal mutation strategy for complex optimization problem, Appl. Soft Comput., 2020 (2020), 106724.
[7]	B. Ko, K. H. Cheong, J. Y. Nam, Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian Networks, Fire Saf. J., 45 (2010), 262-270. doi: 10.1016/j.firesaf.2010.04.001
[8]	M. Mueller, P. Karasev, I. Kolesov, A. Tannenbaum, Optical flow estimation for flame detection in videos, IEEE Trans. Image Process., 22 (2013), 2786-2797. doi: 10.1109/TIP.2013.2258353
[9]	H. Zhao, J. Zheng, W. Deng, Y. Song, Semi-supervised broad learning system based on manifold regularization and broad network, IEEE Trans. Circuits Syst., 67 (2020), 983-994. doi: 10.1109/TCSI.2019.2959886
[10]	W. Deng, H. Liu, J. Xu, H. Zhao, Y. Song, An improved quantum-inspired differential evolution algorithm for deep belief network, IEEE Trans. Instrum. Meas., 69 (2020), 7319-7327. doi: 10.1109/TIM.2020.2983233
[11]	Y. Liu, Y. Mu, K. Chen, Y. Li, J. Guo, Daily activity feature selection in smart homes based on pearson correlation coefficient, Neural Process. Lett., 51 (2020), 1771-1787. doi: 10.1007/s11063-019-10185-8
[12]	R. Chen, S. K. Guo, X. Z. Wang, T. L. Zhang, Fusion of multi-RSMOTE with fuzzy integral to classify bug reports with an imbalanced distribution, IEEE Trans. Fuzzy Syst., 27 (2019), 2406-2420. doi: 10.1109/TFUZZ.2019.2899809
[13]	W. Deng, H. Liu, J. Xu, H. Zhao, Y. Song, An improved quantum-inspired differential evolution algorithm for deep belief network, IEEE Trans. Instrum. Meas., 69 (2020), 7319-7327. doi: 10.1109/TIM.2020.2983233
[14]	Y. Xu, H. Chen, J. Luo, Q. Zhang, S. Jiao, X. Zhang, Enhanced Moth-flame optimizer with mutation strategy for global optimization, Inf. Sci., 492 (2019), 181-203. doi: 10.1016/j.ins.2019.04.022
[15]	Y. Liu, X. Wang, Z. Zhai, R. Chen, Y. Jiang, Timely daily activity recognition from headmost sensor events, ISA Trans., 94 (2019), 379-390. doi: 10.1016/j.isatra.2019.04.026
[16]	W. Deng, J. Xu, H. Zhao, Y. Song, A novel gate resource allocation method using improved PSO-based QEA, IEEE Trans. Intell. Transp. Syst., 2020 (2020), 1-9.
[17]	H. Chen, A. A. Heidari, H. Chen, M. Wang, Z. Pan, A. H. Gandomi, Multi-population differential evolution-assisted Harris hawks optimization: Framework and case studies, Future Gener. Comput. Syst., 111 (2020), 175-198. doi: 10.1016/j.future.2020.04.008
[18]	Y. Xue, B. Xue, M. Zhang, Self-adaptive particle swarm optimization for large-scale feature selection in classification, ACM Trans. Knowl. Discovery Data, 13 (2019), 1-27.
[19]	W. Deng, J. Xu, Y. Song, H. Zhao, An effective improved co-evolution ant colony optimization algorithm with multi-strategies and its application, Int. J. Bio-Inspired Comput., 2019 (2019), 1-10.
[20]	T. H. Chen, P. H. Wu, Y. C. Chiou, An early fire-detection method based on image processing, 2004 International Conference on Image Processing, 3 (2004), 1707-1710.
[21]	A. Fernandez, M. X. Alvarez, F. Bianconi, Texture description through histograms of equivalent patterns, J. Math. Imaging Vision, 45 (2013), 76-102. doi: 10.1007/s10851-012-0349-8
[22]	W. Xu, C. Tian, S. Fang, Fire automatic recognition based on image visual feature, Comput. Eng., 18 (2003), 112-113.
[23]	T. CElik, H. Demirel, Fire detection in video sequences using a generic color model, Fire Saf. J., 44 (2009), 147-158. doi: 10.1016/j.firesaf.2008.05.005
[24]	P. Foggia, A. Saggese, M. Vento, Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion. IEEE Trans. Circuits Syst. Video Technol., 25 (2015), 1545-1556. doi: 10.1109/TCSVT.2015.2392531
[25]	S. Frizzi, R. Kaabi, M. Bouchouicha, J. Ginoux, E. Moreau, F. Fnaiech, Convolutional neural network for video fire and smoke detection, IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society, 2016,877-882.
[26]	T. J. Fu, C. E. Zheng, Y. Tian, Q. M. Qiu, S. J. Lin, Forest fire recognition based on deep convolutional neural network under complex background, Comput. Modernization, 3 (2016), 52-57.
[27]	M. Kang, B. S. Wang, An image filtering method based on image enhancement, Geomatics Inf. Sci. Wuhan Univ., 34 (2009), 822-825.
[28]	Y. Gu, L. J. Qin, L. L. Jiang, Research on PCA and K-SVD joint filtering method, Electro-Optic Technol. Appl., 31 (2016), 31-36+45.
[29]	Y. Q. Zhao, J. Yang, Hyperspectral image denoising via sparse representation and low-rank constraint, IEEE Trans. Geosci. Remote Sens., 53 (2014), 296-308.
[30]	Y. Xu, Z. Wu, Z. Wei, Spectral-spatial classification of hyperspectral image based on low-rank decomposition, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 8 (2015), 2370-2380. doi: 10.1109/JSTARS.2015.2434997
[31]	I. Turkmen, The ANN based detector to remove random-valued impulse noise in images, J. Visual Commun. Image Representation, 34 (2016), 28-36. doi: 10.1016/j.jvcir.2015.10.011
[32]	K. J. Wang, X. Y. Xiong, Z. Ren, Highly efficient mean filtering algorithm, Appl. Res. Comput., 27 (2010), 434-438.
[33]	D. Goyal, M. Singhal, Area-efficient FPGA model of LMS filtering algorithm, Proceedings of the International Conference on Recent Cognizance in Wireless Communication & Image Processing, 2016,943-952.
[34]	B. Hu, Infrared image de-noising based on wavelet transform and improved median filtering, Mod. Electro. Tech., 34 (2011), 50-52.
[35]	W. L. Jiang, G. L. Li, W. B. Luo, Application of improved median filtering algorithm to image de-noising, Adv. Mater. Res., 998 (2014), 838-841.
[36]	H. K. Xu, Y. Y. Qin, H. R. Chen, An improved edge detection algorithm based on Canny, Infrared Technol., 36 (2014), 210-214.
[37]	Q. Zhang, J. Xu, L. Xu, H. Guo, Deep convolutional neural networks for forest fire detection, 2016 International Forum on Management, Education and Information Technology Application, Atlantis Press, 2016.
[38]	Y. Zhao, Z. Zhou, M. Xu, Forest fire smoke video detection using spatiotemporal and dynamic texture features, J. Electr. Comput. Eng., 2015 (2015), 1-7.
[39]	G. F. Shidik, F. N. Adnan, C. Supriyanto, R. A. Pramunendar, P. N. Andono, Multi color feature, background subtraction and time frame selection for fire detection, 2013 International Conference on Robotics, Biomimetics, Intelligent Computational Systems. IEEE, 2013,115-120.
[40]	R. C. Gonzalez, R. E. Woods, Digital image processing (3rd Edition), Prentice-Hall, Inc., 2007.
[41]	Y. Wang, H. Wang, C. Yin, M. Dai, Biologically inspired image enhancement based on Retinex, Neurocomputing, 177 (2016), 373-384. doi: 10.1016/j.neucom.2015.10.124
[42]	J. Chen, Y. He, J. Wang, Multi-feature fusion based fast video flame detection, Build. Environ., 45 (2010), 1113-1122. doi: 10.1016/j.buildenv.2009.10.017
[43]	B. U. Toreyin, Y. Dedeoglu, U. Gudukbay, A. E. Cetin, Computer vision based method for real-time fire and flame detection, Pattern Recognit. Lett., 27 (2006), 49-58. doi: 10.1016/j.patrec.2005.06.015
[44]	Y. J. Hu, Z. F. Li, Y. M. Hu, Theory and application of motion analysis based on optical flow, Comput. Meas. Control, 15 (2007), 219-221.
[45]	H. Zhu, D. Y. Luo, Q. X. Cao, Moving objects detection algorithm based on two consecutive frames subtraction and background subtraction, Comput. Meas. Control, 13 (2005), 215-217.
[46]	T. Zhang, H. Zhang, R. Wang, Y. Wu, A new JPEG image steganalysis technique combining rich model features and convolutional neural networks, Math. BioSci. Eng., 16 (2019), 4069-4081. doi: 10.3934/mbe.2019201
[47]	K. Zhang, W. Zuo, Y. Chen, D. Meng, L. Zhang, Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, IEEE Trans. Image Process., 26 (2016), 3142-3155.
[48]	E. K. Wang, F. Wang, R. Sun, X. Liu, A new privacy attack network for remote sensing images classification with small training samples, Math. BioSci. Eng., 16 (2019), 4456-4476. doi: 10.3934/mbe.2019222
[49]	J. Dunnings, T. P. Breckon, Experimentally defined convolutional neural network architecture variants for non-temporal real-time fire detection, 2018 25th IEEE International Conference on Image Processing (ICIP), 2018, 1558-1562.
[50]	C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al, Going deeper with convolutions, Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, 2015, 1-9.
[51]	C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, 2818-2826.
[52]	J. Zhang, C. Lu, X. Li, H. J. Kim, J. Wang, A full convolutional network based on DenseNet for remote sensing scene classification, Math. Biosci. Eng., 16 (2019), 3345-3367. doi: 10.3934/mbe.2019167
[53]	J. O'Malley, K. H. Zou, Bayesian multivariate hierarchical transformation models for ROC analysis, Stat. Med., 25 (2010), 459-479.
[54]	J. Sharma, O. C. Granmo, M. Goodwin, J. T. Fidje, Deep convolutional neural networks for fire detection in images, International Conference on Engineering Applications of Neural Networks, 2017,183-193.
[55]	A. Fernandez, M. X. Alvarez, F. Bianconi, Texture description through histograms of equivalent patterns, J. Math. Imaging Vision, 45 (2013), 76-102. doi: 10.1007/s10851-012-0349-8
[56]	J. T. Shi, F. N. Yuan, X. Xia, Research progress of video smoke detection, J. Image Graphics, 23 (2018), 303-322.

Reader Comments

Your name:*

Email:*
© 2020 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)