Lightweight network study of leather defect segmentation with Kronecker product multipath decoding

Zhongliang Zhang; Yao Fu; Huiling Huang; Feng Rao; Jun Han; Zhongliang Zhang; Yao Fu; Huiling Huang; Feng Rao; Jun Han

doi:10.3934/mbe.2022642

Mathematical Biosciences and Engineering

2022, Volume 19, Issue 12: 13782-13798. doi: 10.3934/mbe.2022642

Previous Article Next Article

Research article Special Issues

Lightweight network study of leather defect segmentation with Kronecker product multipath decoding

1.
School of Physical and Mathematical Sciences, Nanjing Tech University, Nanjing, Jiangsu 211816, China
2.
Quanzhou Institute of Equipment Manufacturing, Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences, Quanzhou, Fujian 362201, China

Academic Editor: Jerry Chun-Wei Lin

Received: 03 August 2022 Revised: 04 September 2022 Accepted: 15 September 2022 Published: 19 September 2022

In the leather production process, defects on the leather surface are a key factor in the quality of the finished leather. Leather defect detection is an important step in the leather production process, especially for wet blue leather. To improve the efficiency and accuracy of detection, we propose a leather segmentation network using the Kronecker product for multi-path decoding and named KMDNet. The network uses Kronecker products to construct a new semantic information extraction layer named KPCL layer. The KPCL layer is added to the decoding network to form new decoding paths, and these different decoding paths are combined that segment the defective part of the leather image. We collaborate with leather companies to collect relevant leather defect images; use Tensorflow for training, validation, and testing experiments; and compare the detection results with non-machine learning algorithms and semantic segmentation algorithms. The experimental results show that KMDNet has a $ 1.99\% $ improvement in $ F1 $ score compared to UNet for leather and a nearly three times improvement in detection speed.
- wet blue leather defect,
- semantic segmentation,
- multi-path decoding,
- Kronecker product,
- feature fusion
Citation: Zhongliang Zhang, Yao Fu, Huiling Huang, Feng Rao, Jun Han. Lightweight network study of leather defect segmentation with Kronecker product multipath decoding[J]. Mathematical Biosciences and Engineering, 2022, 19(12): 13782-13798. doi: 10.3934/mbe.2022642

Related Papers:

Abstract

In the leather production process, defects on the leather surface are a key factor in the quality of the finished leather. Leather defect detection is an important step in the leather production process, especially for wet blue leather. To improve the efficiency and accuracy of detection, we propose a leather segmentation network using the Kronecker product for multi-path decoding and named KMDNet. The network uses Kronecker products to construct a new semantic information extraction layer named KPCL layer. The KPCL layer is added to the decoding network to form new decoding paths, and these different decoding paths are combined that segment the defective part of the leather image. We collaborate with leather companies to collect relevant leather defect images; use Tensorflow for training, validation, and testing experiments; and compare the detection results with non-machine learning algorithms and semantic segmentation algorithms. The experimental results show that KMDNet has a $ 1.99\% $ improvement in $ F1 $ score compared to UNet for leather and a nearly three times improvement in detection speed.

References

[1]	C. Kwaka, J. A. Venturab, K. Tofang-Sazi, Automated defect inspection andclassification of leather fabric, Intell. Data Anal., 5 (2001), 355–370. https://doi.org/10.3233/IDA-2001-5406 doi: 10.3233/IDA-2001-5406
[2]	F. A. Faiz, A. Azhari, Tanned and synthetic leather classification based on images texture with convolutional neural network, Knowl. Eng. Data Sci., 3 (2020), 77–88. http://dx.doi.org/10.17977/um018v3i22020p77-88 doi: 10.17977/um018v3i22020p77-88
[3]	Y. T. Lee, C. Yeh, Automatic recognition and defect compensation for calf leather, Int. J. Inf. Technol. Manage., 19 (2020), 93–117. https://doi.org/10.1504/IJITM.2020.106211 doi: 10.1504/IJITM.2020.106211
[4]	M. Jawahar, L. J. Anbarasi, S. G. Jasmine, M. Narendra, R. Venba, V. Karthik, A machine learning-based multi-feature extraction method for leather defect classification, in Inventive Computation and Information Technologies, 173 (2021), 189–202. https://doi.org/10.1007/978-981-33-4305-4_15
[5]	Y. S. Gan, S. T. Liong, S. Y. Wang, C. T. Cheng, An improved automatic defect identification system on natural leather via generative adversarial network, Int. J. Computer Integr. Manuf., 2022 (2022), 1–17. https://doi.org/10.1080/0951192X.2022.2048421 doi: 10.1080/0951192X.2022.2048421
[6]	Y. S. Gan, W. C. Yau, S. T. Liong, C. C. Che, Automated classification system for tick-bite defect on leather, Math. Probl. Eng., 2022 (2022), 5549879. https://doi.org/10.1155/2022/5549879 doi: 10.1155/2022/5549879
[7]	T. Adao, D. Gonzalez, Y. C. Castilla, J. Perez, S. Shahrabadi, N. Sousa, et al., Using deep learning to detect the presence/absence of defectson leather: on the way to build an industry-driven approach, J. Phys. Conf. Ser., 2224 (2022), 012009. https://doi.org/10.1088/1742-6596/2224/1/012009 doi: 10.1088/1742-6596/2224/1/012009
[8]	F. López, J. M. Prats, A. Ferrer, J. M. Valiente, Defect detection in random colour textures using the MIA T$^{2}$ defect maps, in ICIAR 2006: Image Analysis and Recognition, (2006), 752–763. https://doi.org/10.1007/11867661_68
[9]	R. Viana, R. B. Rodrigues, M. A. Alvarez, H. Pistori, SVM with stochastic parameter selection for bovine leather defect classification, in PSIVT 2007: Advances in Image and Video Technology, (2007), 600–612. https://doi.org/10.1007/978-3-540-77129-6_52
[10]	H. Q. Bong, Q. B. Truong, H. C. Nguyen, M. T. Nguyen, Vision-based inspection system for leather surface defect detection and classification, in 2018 5th NAFOSTED Conference on Information and Computer Science (NICS), (2018), 300–304. https://doi.org/10.1109/NICS.2018.8606836
[11]	C. Kwak, J. A. Ventura, K. Tofang-Sazi, A neural network approach for defect identification and classification on leather fabric, J. Intell. Manuf., 11 (2000), 485–499. https://doi.org/10.1023/A:1008974314490 doi: 10.1023/A:1008974314490
[12]	A. Varghese, S. Jain, A. A. Prince, M. Jawahar, Digital microscopic image sensing and processing for leather species identification, IEEE Sens. J., 20 (2020), 10045–10056. https://doi.org/10.1109/JSEN.2020.2991881 doi: 10.1109/JSEN.2020.2991881
[13]	D. M. Tsai, T. Y. Huang, Automated surface inspection for statistical textures, Image Vision Comput., 21 (2003), 307–323. https://doi.org/10.1016/S0262-8856(03)00007-6 doi: 10.1016/S0262-8856(03)00007-6
[14]	J. W. Kwon, Y. Y. Choo, H. H. Choi, J. M. Cho, G. S. KiI, Development of leather quality discrimination system by texture analysis, in 2004 IEEE Region 10 Conference TENCON 2004, 1 (2004), 327–330. https://doi.org/10.1109/TENCON.2004.1414423
[15]	K. Krastev, L. Georgieva, Identification of leather surface defects using fuzzy logic, in 2005 International Conference on Computer Systems and Technologies, (2005), IIIA.12-1–IIIA.12-6. Available from: http://ecet.ecs.uni-ruse.bg/cst05/Docs/cp/SIII/IIIA.12.pdf.
[16]	D. H. Fan, L. Ding, J. H. Deng, Automatic detection and localization of surface defects for whole piece of ultrahigh-definition leather images, in 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS), (2019), 229–232. https://doi.org/10.1109/CCOMS.2019.8821662
[17]	S. T. Liong, D. Zheng, Y. C. Huang, Y. S. Gan, Leather defect classification and segmentation using deep learning architecture, Int. J. Computer Integr. Manuf., 33 (2020), 1105–1117. https://doi.org/10.1080/0951192X.2020.1795928 doi: 10.1080/0951192X.2020.1795928
[18]	J. Wang, G. Yi, S. Zhang, Y. Wang, An unsupervised generative adversarial network-based method for defect inspection of texture surfaces, Appl. Sci., 11 (2020), 283. https://doi.org/10.3390/app11010283 doi: 10.3390/app11010283
[19]	S. Y. Chen, Y. C. Cheng, W. L. Yang, M. Y. Wang, Surface defect detection of wet-blue leather using hyperspectral imaging, IEEE Access, 9 (2021), 127685–127702. https://doi.org/10.1109/ACCESS.2021.3112133 doi: 10.1109/ACCESS.2021.3112133
[20]	Y. Shen, T. Xiao, S. Yi, D. Chen, X. Wang, H. Li, Person re-identification with deep kronecker-product matching and group-shuffling random walk, IEEE Trans. Pattern Anal. Mach. Intell., 43 (2019), 1649–1665. https://doi.org/10.1109/TPAMI.2019.2954313 doi: 10.1109/TPAMI.2019.2954313
[21]	Z. J. Xiao, X. D. Yang, X. Wei, X. L. Tang, Improved lightweight network in image recognition, J. Front. Comput. Sci. Technol., 15 (2021), 743–753. https://doi.org/10.3778/j.issn.1673-9418.2004057 doi: 10.3778/j.issn.1673-9418.2004057
[22]	H. V. Henderson, F. Pukelsheim, S. R. Searle, On the history of the Kronecker product, Linear Multilinear Algebra, 14 (1983), 113–120. https://doi.org/10.1080/03081088308817548 doi: 10.1080/03081088308817548
[23]	T. Wu, S. Tang, R. Zhang, J. Cao, J. Li, Tree-structured kronecker convolutional network for semantic segmentation, in 2019 IEEE International Conference on Multimedia and Expo (ICME), (2019), 940–945. https://doi.org/10.1109/ICME.2019.00166
[24]	F. Yu, V. Koltun, Multi-scale context aggregation by dilated convolutions, preprint, arXiv: 1511.07122.
[25]	K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, arXiv: 1409.1556.
[26]	O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, (2015), 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
[27]	V. Badrinarayanan, A. Kendall, R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615 doi: 10.1109/TPAMI.2016.2644615
[28]	H. S. Zhao, J. P. Shi, X. J. Qi, X. G. Wang, J. Y. Jia, Pyramid scene parsing network, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 2881–2890. https://doi.org/10.1109/CVPR.2017.660
[29]	H. C. Li, P. F. Xiong, H. Q. Fan, J. Sun, Dfanet: Deep feature aggregation for real-time semantic segmentation, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019), 9522–9531.

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)