The automatic surface defect detection system supports the real-time surface defect detection by reducing the information and high-lighting the critical defect regions for high level image under-standing. However, the defects exhibit low contrast, different textures and geometric structures, and several defects making the surface defect detection more difficult. In this paper, a pixel-wise detection framework based on convolutional neural network (CNN) for strip steel surface defect detection is proposed. First we extract the salient features by a pre-trained backbone network. Secondly, contextual weighting module, with different convolutional kernels, is used to extract multi-scale context features to achieve overall defect perception. Finally, the cross integrate is employed to make the full use of these context information and decoded the information to realize feature information complementation. The experimental results of this study demonstrate that the proposed method outperforms against the previous state-of-the-art methods on strip steel surface defect dataset (MAE: 0.0396; Fβ: 0.8485).
Citation: Guozhen Dong. A pixel-wise framework based on convolutional neural network for surface defect detection[J]. Mathematical Biosciences and Engineering, 2022, 19(9): 8786-8803. doi: 10.3934/mbe.2022408
The automatic surface defect detection system supports the real-time surface defect detection by reducing the information and high-lighting the critical defect regions for high level image under-standing. However, the defects exhibit low contrast, different textures and geometric structures, and several defects making the surface defect detection more difficult. In this paper, a pixel-wise detection framework based on convolutional neural network (CNN) for strip steel surface defect detection is proposed. First we extract the salient features by a pre-trained backbone network. Secondly, contextual weighting module, with different convolutional kernels, is used to extract multi-scale context features to achieve overall defect perception. Finally, the cross integrate is employed to make the full use of these context information and decoded the information to realize feature information complementation. The experimental results of this study demonstrate that the proposed method outperforms against the previous state-of-the-art methods on strip steel surface defect dataset (MAE: 0.0396; Fβ: 0.8485).
| [1] |
K. Song, Y. Yan, A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects, Appl. Surf. Sci., 285 (2013), 858-864. https://doi.org/10.1016/j.apsusc.2013.09.002 doi: 10.1016/j.apsusc.2013.09.002
|
| [2] |
Y. Liu, K. Xu, D. Wang, Online surface defect identification of cold rolled strips based on local binary pattern and extreme learning machine, Metals, 8 (2018), 197. https://doi.org/10.3390/met8030197 doi: 10.3390/met8030197
|
| [3] | D. Djukic, S. Spuzic, Statistical discriminator of surface defects on hot rolled steel, Image Vis. Comput., (2007), 158-163. |
| [4] |
J. Wang, Q. Li, J. Gan, H. Yu, X. Yang, Surface defect detection via entity sparsity pursuit with intrinsic priors, IEEE Trans. Ind. Inform., 16 (2019), 141-150. https://doi.org/10.1109/TII.2019.2917522 doi: 10.1109/TII.2019.2917522
|
| [5] |
N. Neogi, D. K. Mohanta, P. K. Dutta, Defect detection of steel surfaces with global adaptive percentile thresholding of gradient image, J. Institut. Eng. (India) Series B, 98 (2017), 557-565. https://doi.org/10.1007/s40031-017-0296-2 doi: 10.1007/s40031-017-0296-2
|
| [6] |
D. C. Choi, Y. J. Jeon, S. H. Kim, S. Moon, J. P. Yun, S. W. Kim, Detection of pinholes in steel slabs using Gabor filter combination and morphological features, ISIJ Int., 57 (2017), 1045-1053. https://doi.org/10.2355/isijinternational.ISIJINT-2016-160 doi: 10.2355/isijinternational.ISIJINT-2016-160
|
| [7] |
X. Xie, M. Mirmehdi, TEXEMS: Texture exemplars for defect detection on random textured surfaces, IEEE. Trans. Pattern Anal. Mach. Intell., 29 (2007), 1454-1464. https://doi.org/10.1109/TPAMI.2007.1038 doi: 10.1109/TPAMI.2007.1038
|
| [8] |
F. S. Cohen, Z. Fan, S. Attali, Automated inspection of textile fabrics using textural models, IEEE. Trans. Pattern Anal. Mach. Intell., 13 (1991), 803-808. https://doi.org/10.1109/34.85670 doi: 10.1109/34.85670
|
| [9] | Y. He, K. Song, H. Dong, Y. Yan, Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network, Opt. Lasers Eng., 122,294-302. https://doi.org/10.1016/j.optlaseng.2019.06.020 |
| [10] |
V. Natarajan, T. Y. Hung, S. Vaikundam, L. T. Chia, Convolutional networks for voting-based anomaly classification in metal surface inspection, IEEE International Conference on Industrial Technology (ICIT), (2017), 986-991. https://10.1109/ICIT.2017.7915495 doi: 10.1109/ICIT.2017.7915495
|
| [11] |
M. Win, A. R. Bushroa, M. A. Hassan, N. M. Hilman, A. Ide-Ektessabi, A contrast adjustment thresholding method for surface defect detection based on mesoscopy, IEEE Trans. Ind. Inform., 11 (2015), 642-649. https://doi.org/10.1109/TⅡ.2015.2417676 doi: 10.1109/TII.2015.2417676
|
| [12] |
M. Ricci, A. Ficola, M. Fravolini, L. Battaglini, A. Palazzi, P. Burrascano, et al., Magnetic imaging and machine vision NDT for the on-line inspection of stainless steel strips, Meas. Sci. Technol., 24 (2012), 025401. https://doi.org/10.1007/s11276-012-0479-3 doi: 10.1007/s11276-012-0479-3
|
| [13] |
H. Hu, Y. Liu, M. Liu, L. Nie, Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm, Neurocomputing, 181 (2016), 86-95. https://doi.org/10.1016/j.neucom.2015.05.134 doi: 10.1016/j.neucom.2015.05.134
|
| [14] |
Y. J. Zhao, Y. H. Yan, K. C. Song, Vision-based automatic detection of steel surface defects in the cold rolling process: considering the influence of industrial liquids and surface textures, Int. J. Adv. Manuf. Technol., 90 (2017), 1665-1678. https://doi.org/10.1007/s00170-016-9489-0 doi: 10.1007/s00170-016-9489-0
|
| [15] |
Y. Wang, H. Xia, X. Yuan, L. Li, B. Sun, Distributed defect recognition on steel surfaces using an improved random forest algorithm with optimal multi-feature-set fusion, Multimed. Tools Appl., 77 (2018), 16741-16770. https://doi.org/10.1007/s11042-017-5238-0 doi: 10.1007/s11042-017-5238-0
|
| [16] | M. Chu, R. Gong, S. Gao, J. Zhao, Steel surface defects recognition based on multi-type statistical features and enhanced twin support vector machine, Chemometrics Intell. Lab. Syst., 171,140-150. https://doi.org/10.1016/j.chemolab.2017.10.020 |
| [17] |
S. Fekri-Ershad, F. Tajeripour, Multi-resolution and noise-resistant surface defect detection approach using new version of local binary patterns, Appl. Artif. Intell., 31 (2017), 395-410. https://doi.org/10.1080/08839514.2017.1378012 doi: 10.1080/08839514.2017.1378012
|
| [18] |
X. Zhang, W. Li, J. Xi, Z. Zhang, X. Fan, Surface defect target identification on copper strip based on adaptive genetic algorithm and feature saliency, Math. Probl. Eng., 2013. https://doi.org/10.1155/2013/504895 doi: 10.1155/2013/504895
|
| [19] |
Y. H. Ai, K. Xu, Surface detection of continuous casting slabs based on curvelet transform and kernel locality preserving projections, J. Iron Steel Res. Int., 20 (2013), 80-86. https://doi.org/10.1016/S1006-706X(13)60102-8 doi: 10.1016/S1006-706X(13)60102-8
|
| [20] |
Ş. Öztürk, B. Akdemir, Real-time product quality control system using optimized Gabor filter bank, Int. J. Adv. Manuf. Technol., 96 (2018), 11-19. https://doi.org/10.1007/s00170-018-1585-x doi: 10.1007/s00170-018-1585-x
|
| [21] |
D. C. Choi, Y. J. Jeon, S. J. Lee, J. P. Yun, S. W. Kim, Algorithm for detecting seam cracks in steel plates using a Gabor filter combination method, Appl. optics, 53 (2014), 4865-4872. https://doi.org/10.1364/AO.53.004865 doi: 10.1364/AO.53.004865
|
| [22] |
X. Y. Wu, K. Xu, J. W. Xu, Application of undecimated wavelet transform to surface defect detection of hot rolled steel plates, In 2008 Congress on Image and Signal Processing, (2008), 528-532. https://10.1109/CISP.2008.278 doi: 10.1109/CISP.2008.278
|
| [23] |
Ş. Öztürk, B. Akdemır, Novel BiasFeed cellular neural network model for glass defect inspection, In 2016 International Conference on Control, Decision and Information Technologies (CoDIT), (2016), 366-371. https://doi.org/10.1109/CoDIT.2016.7593590 doi: 10.1109/CoDIT.2016.7593590
|
| [24] |
X. Li, S. K. Tso, X. P. Guan, Q. Huang, Improving automatic detection of defects in castings by applying wavelet technique, IEEE Trans. Ind. Electron., 53 (2006), 1927-1934. https://doi.org/10.1109/TIE.2006.885448 doi: 10.1109/TIE.2006.885448
|
| [25] |
X. Liu, K. Xu, P. Zhou, D. Zhou, Y. Zhou, Surface defect identification of aluminium strips with non-subsampled shearlet transform, Opt. Lasers Eng., (2020). https://doi.org/10.1016/j.optlaseng.2019.105986 doi: 10.1016/j.optlaseng.2019.105986
|
| [26] |
B. Akdemir, S. Öztürk, Glass surface defects detection with wavelet transforms, Int. J. Mater., Mechan. Manuf., 3 (2015), 170-173. https://doi.org/10.7763/IJMMM.2015.V3.189 doi: 10.7763/IJMMM.2015.V3.189
|
| [27] |
F. S. Cohen, Z. Fan, S. Attali, Automated inspection of textile fabrics using textural models, IEEE. Trans. Pattern Anal. Mach. Intell., 13 (1991), 803-808. https://doi.org/10.1109/34.85670 doi: 10.1109/34.85670
|
| [28] |
G. Song, K. Song, Y. Yan, Saliency detection for strip steel surface defects using multiple constraints and improved texture features, Opt. Lasers Eng., 2019. https://doi.org/10.1016/j.optlaseng.2019.106000 doi: 10.1016/j.optlaseng.2019.106000
|
| [29] | J. Masci, U. Meier, G. Fricout, J. Schmidhuber, Multi-scale pyramidal pooling network for generic steel defect classification, In The 2013 International Joint Conference on Neural Networks (IJCNN), 2013. https://doi.org/10.1109/IJCNN.2013.6706920 |
| [30] |
D. He, K. Xu, P. Zhou, Defect detection of hot rolled steels with a new object detection framework called classification priority network, Comput. Ind. Eng., 128 (2018), 290-297. https://doi.org/10.1016/j.cie.2018.12.043 doi: 10.1016/j.cie.2018.12.043
|
| [31] |
Y. He, K. Song, Q. Meng, Y. Yan, An end-to-end steel surface defect detection approach via fusing multiple hierarchical features, IEEE Trans. Instrum. Meas., 69 (2019), 1493-1504. https://doi.org/10.1109/TIM.2019.2915404 doi: 10.1109/TIM.2019.2915404
|
| [32] |
X. Kou, S. Liu, K. Cheng, Y. Qian, Development of a YOLO-V3-based model for detecting defects on steel strip surface, Measurement, 182 (2021). https://doi.org/10.1016/j.measurement.2021.109454 doi: 10.1016/j.measurement.2021.109454
|
| [33] |
R. Ren, T. Hung, K. C. Tan, A generic deep-learning-based approach for automated surface inspection, IEEE T. Cybern., 48 (2017), 929-940. https://doi.org/10.1109/TCYB.2017.2668395 doi: 10.1109/TCYB.2017.2668395
|
| [34] |
H. Yang, Y. Chen, K. Song, Z. Yin, Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects, IEEE Trans. Autom. Sci. Eng., 16 (2019), 1450-1467. https://doi.org/10.1109/TASE.2018.2886031 doi: 10.1109/TASE.2018.2886031
|
| [35] |
R. Neven, T. Goedemé, A multi-branch U-Net for steel surface defect type and severity segmentation, Metals, 11 (2021), 870. https://doi.org/10.3390/met11060870 doi: 10.3390/met11060870
|
| [36] |
X. Zhou, H. Fang, X. Fei, R. Shi, J. Zhang, Edge-aware multi-level interactive network for salient object detection of strip steel surface defects, IEEE Access, (2021). https://doi.org/10.1109/ACCESS.2021.3124814 doi: 10.1109/ACCESS.2021.3124814
|
| [37] |
G. Song, K. Song, Y. Yan, EDRNet: Encoder-decoder residual network for salient object detection of strip steel surface defects, IEEE Trans. Instrum. Meas., 69 (2020), 9709-9719. https://doi.org/10.1109/TIM.2020.3002277 doi: 10.1109/TIM.2020.3002277
|
| [38] |
H. Dong, K. Song, Y. He, J. Xu, Y. Yan, Q. Meng, PGA-Net: Pyramid feature fusion and global context attention network for automated surface defect detection, IEEE Trans. Ind. Inform., 16 (2019), 7448-7458. https://doi.org/10.1109/TII.2019.2958826 doi: 10.1109/TII.2019.2958826
|
| [39] | K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint, (2014), arXiv: 1409.1556. https://doi.org/10.48550/arXiv.1409.1556 |
| [40] |
J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (2015), 3431-3440. https://doi.org/10.1109/CVPR.2015.7298965 doi: 10.1109/CVPR.2015.7298965
|
| [41] |
O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, In International Conference on Medical image computing and computer-assisted intervention, (2015), 234-241. https://doi.org/10.1007/978-3-319-24574-4_28 doi: 10.1007/978-3-319-24574-4_28
|
| [42] |
H. J. Kim, E. Dunn, J. M. Frahm, Learned contextual feature reweighting for image geo-localization, In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 3251-3260. https://doi.org/10.1109/CVPR.2017.346 doi: 10.1109/CVPR.2017.346
|
| [43] |
P. T. De Boer, D. P. Kroese, S. Mannor, R. Y. Rubinstein, A tutorial on the cross-entropy method, Ann. Oper. Res., 134 (2005), 19-67. https://doi.org/10.1007/s10479-005-5724-z doi: 10.1007/s10479-005-5724-z
|
| [44] |
M. A. Rahman, Y. Wang, Optimizing intersection-over-union in deep neural networks for image segmentation, In International symposium on visual computing, (2016), 234-244. https://doi.org/10.1007/978-3-319-50835-1_22 doi: 10.1007/978-3-319-50835-1_22
|
| [45] |
Z. Wang, E. P. Simoncelli, A. C. Bovik, Multiscale structural similarity for image quality assessment, In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2 (2003), 1398-1402. https://doi.org/10.1109/ACSSC.2003.1292216 doi: 10.1109/ACSSC.2003.1292216
|
| [46] |
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, et al., TensorFlow: A system for large-scale machine learning, In 12th USENIX symposium on operating systems design and implementation (OSDI 16), (2016), 265-283. https://dl.acm.org/doi/10.5555/3026877.3026899 doi: 10.5555/3026877.3026899
|
| [47] |
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Adv. Neural Inform. Process. Syst., 2017. https://doi.org/10.1145/3065386 doi: 10.1145/3065386
|
| [48] |
F. Perazzi, P. Krähenbühl, Y. Pritch, A. Hornung, Saliency filters: Contrast based filtering for salient region detection, In 2012 IEEE conference on computer vision and pattern recognition (CVPR), (2012), 733-740. https://doi.org/10.1109/CVPR.2012.6247743 doi: 10.1109/CVPR.2012.6247743
|
| [49] |
Y. Qin, H. Lu, Y. Xu, H. Wang, Saliency detection via cellular automata, In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (2012), 110-119. https://doi.org/10.1109/CVPR.2012.6247743 doi: 10.1109/CVPR.2012.6247743
|
| [50] |
R. Achanta, S. Hemami, F. Estrada, S. Susstrunk, Frequency-tuned salient region detection, In 2009 IEEE conference on computer vision and pattern recognition (CVPR), (2012), 1597-1604. https://doi.org/10.1109/CVPR.2009.5206596 doi: 10.1109/CVPR.2009.5206596
|
| [51] |
F. Huang, J. Qi, H. Lu, L. Zhang, X. Ruan, Salient object detection via multiple instance learning, IEEE Trans. Image Process., 26 (2017), 1911-1922. https://doi.org/10.1109/TIP.2017.2669878 doi: 10.1109/TIP.2017.2669878
|
| [52] |
M. M. Cheng, N. J. Mitra, X. Huang, P. H. Torr, S. M. Hu, Global contrast based salient region detection, IEEE. Trans. Pattern Anal. Mach. Intell., 37 (2014), 569-582. https://doi.org/10.1109/TPAMI.2014.2345401 doi: 10.1109/TPAMI.2014.2345401
|
| [53] |
H. Peng, B. Li, H. Ling, W. Hu, W. Xiong, S. J. Maybank, Salient object detection via structured matrix decomposition, IEEE. Trans. Pattern Anal. Mach. Intell., 39 (2016), 818-832. https://doi.org/10.1109/TPAMI.2016.2562626 doi: 10.1109/TPAMI.2016.2562626
|
| [54] | H. Noh, S. Hong, B. Han, Learning deconvolution network for semantic segmentation, In Proceedings of the IEEE international conference on computer vision, (2015), 1520-1528. https://doi.org/10.1109/ICCV.2015.178 |
| [55] |
N. Liu, J. Han, Dhsnet: Deep hierarchical saliency network for salient object detection, In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (2016), 678-686. https://doi.org/10.1109/CVPR.2016.80 doi: 10.1109/CVPR.2016.80
|
| [56] |
Q. Hou, M. M. Cheng, X. Hu, A. Borji, Z. Tu, P. H. Torr, Deeply supervised salient object detection with short connections, In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (2016), 3203-3212. https://doi.org/10.1109/TPAMI.2018.2815688 doi: 10.1109/TPAMI.2018.2815688
|
Figures(7) / Tables(3)
Guozhen Dong. A pixel-wise framework based on convolutional neural network for surface defect detection[J]. Mathematical Biosciences and Engineering, 2022, 19(9): 8786-8803. doi: 10.3934/mbe.2022408
DownLoad: