A feature extraction and classification algorithm based on improved sparse auto-encoder for round steel surface defects

1.   Introduction

Round steel is a typical steel product, widely used in aerospace, transportation, civil and industrial construction, equipment manufacturing, marine engineering and other fields ^[1]. In the process of round steel production, due to the equipment failure, product material changes, product process fluctuations and other factors, the final products will inevitably have some surface quality problems, such as scratches, ears, cracks, scars, fold over, roll marks, etc ^[2]. Machine vision based defects detection has become one of the important technologies for modern steel enterprises to solve the product surface quality problems ^[3,4,5,6,7], which mainly includes image acquisition, image preprocessing, defect segmentation, defect feature extraction and classification ^[8]. Among all of them, an efficient and accurate defect feature extraction algorithm is core of the technology, so that enterprises can quickly and accurately detect the surface defects, adjust equipment parameters and optimize production processes.

In general, the defect features of steel surface mainly include color features ^{[9,10,11,12,13]}, texture features ^{[14,15,16,17,18]} and shape features ^{[19,20,21,22,23]}. However, not all of the features are valuable for defects classification. Too many features will cause high dimensions of the classifier, which will increase network training time and reduce classification accuracy. On this basis, the feature dimensionality reduction (FDR) algorithms are adopted to find the optimal features from original image features, so as to reduce the complexity of classification and improve classification accuracy ^[24,25]. Principal component analysis (PCA), kernel principal component analysis (KPCA) and linear discriminant analysis (LDA) are the three commonly used traditional FDR algorithms. These FDR algorithms reduce the feature dimensions in particular theories, but all have certain limitations. The main one is, some useful information may be ignored, which will affect classification accuracy.

Deep learning can also be used for feature extraction ^{[26,27,28,29,30,31,32]}. It is s special type of machine learning and a kind of artificial intelligence ^[33]. With the development of deep learning, some scholars have begun to study the use of deep learning for steel surface defect detection. For example, Wang et al. ^[34] proposed a method of strip surface defect detection method based on deep learning. It took Resnet101 for feature extraction, then used the transfer learning method to train the network to ensure the stability and convergence of the network. Yi et al. ^[35] proposed a strip surface defect recognition system based on deep CNNs. It used original defect image as input and defect category as output for seven main classes of steel strip defects classification. In the practical application, due to the high dimensions of round steel surface defects, the network training of deep learning requires large amount of computation, which will increase the costs of enterprises.

In view of the above problems and causes, this paper proposed a feature extraction and classification algorithm based on improved sparse AE. It first selected the original feature set of round steel surface defects by the statistical features of the image, then adopts three traditional FDR algorithms at the same time to reduce the feature dimensions, combines these defect features obtained in pairs, takes the merged defect features as the input of sparse AE, and uses the “bottleneck” of sparse AE to conduct classification by Softmax. In order to verify the feasibility and effectiveness of proposed algorithms, we take four batches of round steel surface images collected on the actual production line as the test set, compare the defect classification accuracy of proposed algorithm with individual traditional FDR algorithm. As the consequence, the advantages of proposed algorithm can be proved.

This paper is organized as follows: in Section 1, we give an overview of the research background and related works. In Section 2, we describe six typical round steel surface defects and their main image features, on this basis, the initial feature set is selected. In Section 3, we introduce the traditional FDR algorithms: PCA, KPCA and LDA and deep learning FDR algorithm: sparse AE, and their limitations in feature extraction. Our proposed algorithms are given in Section 4, which include: the feature extraction and classification algorithm based on PCA, KPCA and sparse AE (PKAE), the feature extraction and classification algorithm based on PCA, LDA and sparse AE (PLAE) and the feature extraction and classification algorithm based on KPCA, LDA and sparse AE (KLAE). The experimental results are provided in Section 5, in which the algorithm with best classification accuracy and minimum network training time are selected as optimal algorithm, and the feasibility and effectiveness of the optimal algorithm are verified in an actual production line. Finally, concluding remarks and future work are presented in Section 6.

2.   The initial feature set of round steel surface defects

2.1.   Six typical round steel surface defects

The six typical round steel surface defects we selected in this research are: scratches, ears, cracks, scars, fold over and roll marks. The sample images of each defect and their main image features are shown as follows:

2.2.   Initial feature set of round steel surface defects

When using the traditional FDR algorithms to extract defect features, it is often based on prior knowledge to select a reasonable set of features as the initial feature set. On the basis of Section 2.1, the following features are selected as the initial feature set of round steel surface defects:

2.2.1.   Shape features

● Perimeter

Perimeter of the defect is the length of boundary line between the defect area and the background. For binary image, if 1 is the target and 0 is the background, its perimeter is the number of pixels with a boundary value of 1. The calculation formula is as follows:

● Area

Area of the defect is the number of pixels occupied by the defect. For binary image, if 1 is the target and 0 is the background, its area is the number of pixels with a value of 1 in the defect part. The calculation formula is as follows:

● Centroid

Centroid of the defect is the center point of the defect. The calculation formula is as follows:

● Compactness

Compactness represents the shape complexity of the defect. The smaller the value is, the simpler the defect shape is. The calculation formula is as follows:

● Linearity

Linearity is the maximum length of the defect measured from all directions. The smaller the value is, the more linear distribution the defect is. The calculation formula is as follows:

● Hu’s moment invariants

Hu’s moment invariants are the features with translation, rotation and size invariance. For image $f(x, y)$, the $p + q$ moment is defined as:

The central moment is defined as:

where $\overline x$ and $\overline y$ are the central coordinates of the defect: $\overline x = \frac{{{m_{10}}}}{{{m_{00}}}}$, $\overline y = \frac{{{m_{01}}}}{{{m_{00}}}}$.

The central moment can satisfy translation invariance in affine transformation. The normalized central moment formula of $f(x, y)$ is:

where $r = \frac{{q + p}}{2} + 1$.

Therefore, 7invariant matrices m1~m7 can be constructed by using the second and third order normalized central moments:

where ${\eta _{20}}$ represents the extension range of the defect in the horizontal direction, ${\eta _{{\rm{02}}}}$ represents the extension range of the defect in the vertical direction, ${\eta _{{\rm{11}}}}$ represents the inclination of the defect, ${\eta _{{\rm{30}}}}$ represents the degree of shift of the defect gravity center in the horizontal direction, ${\eta _{{\rm{03}}}}$ represents the degree of shift of the defect gravity center in the vertical direction, ${\eta _{{\rm{12}}}}$ represents the degree of equilibrium of the vertical extension range of the defect, ${\eta _{{\rm{21}}}}$ represents the degree of equilibrium of the horizontal extension range of the defect.

2.2.2.   Gray-scale features

The probability density function can be used to represent the gray-scale distribution of defect image. The first order probability distribution of image gray-scale is defined as:

where b is the quantization level, L level in total.$F(x, y)$ is the gray-scale level of the pixel $(x, y)$. Therefore, the first-order histogram is defined as:

where $N(b)$ is the total number of pixels with gray-scale value b, M is the total number of pixels. The gray-scale features obtained from the gray-scale histogram can describe the defect area of the defect image as a whole. On this basis, the gray-scale features selected in this research are:

● Gray-scale mean

Gray-scale mean represents the overall gray-scale level of defect area. The calculation formula is as follows:

● Gray-scale variance

Gray-scale variance represents the discrete degree of the gray-scale distribution of the defect area. The calculation formula is as follows:

● Gray-scale skewness

Gray-scale skewness represents the degree of asymmetry of the gray-scale histogram between different defects. The calculation formula is as follows:

● Gray-scale kurtosis

Gray-scale kurtosis represents the density of the gray-scale histogram around the gray-scale mean. The calculation formula is as follows:

● Gray-scale energy

Gray-scale energy is used to measure the amount of information in the image. When the probability of gray-scale distribution of the defect image is equal, the minimum value is taken. The calculation formula is as follows:

● Gray-scale entropy

Gray-scale entropy is also used to measure the amount of information in the image, which represents the amount of information contained in the agammaegation feature of gray-scale distribution in the defect image. When the probability distribution of defect image is equal, it reaches the maximum value. The calculation formula is as follows:

2.2.3.   Texture features

Gray-scale co-occurrence matrix (GLCM) is an effective method for texture feature selection. It starts from the pixel $(x, y)$ whose gray-scale level is i, and counts the probability $p(i, j, \delta, \theta)$ of its simultaneous occurrence with the pixel whose distance is $\delta$, direction is $\theta$() and gray-scale level is j$\left({x + \Delta x, y + \Delta y} \right)$. The expression of GLCM is:

where $x = 0, 1, 2, \ldots, {N_x} - 1$, $y = 0, 1, 2, \ldots, {N_y} - 1$, $i, j = 0, 1, 2, \ldots, L - 1$. L is the gray-scale level of the image; $x, y$ is the pixel coordinate in the image; ${N_X}$ and ${N_y}$ are the number of rows and columns of the image respectively.

According to GLCM, a large number of texture features can be defined. In this case, the following 4 most commonly used features are selected in this research:

● Texture energy

Texture energy represents the texture roughness of the defect image. The calculation formula is as follows:

where $p(i, j)$ expresses the joint probabilities between two pixels with gray level i and j in distance $\delta$ and direction $\theta$.

● Texture inertia

Texture inertia represents the texture depth of the defect image. The larger the value is, the deeper the texture is. The calculation formula is as follows:

● Texture entropy

Texture entropy represents the complexity of the texture in the defect image. The larger the value is, the greater the randomness of texture is. The calculation formula is as follows:

● Texture correlation

Texture correlation represents the texture direction of the defect image. The larger the value is, the stronger the texture is in this direction than in other directions. The calculation formula is as follows:

where μ_x and σ_x represent the mean value and standard deviation of gray-scale rows; μ_y and σ_y represent the mean value and standard deviation of gray-scale columns.

GLCM only represents the comprehensive information of the defect image in certain range and angle of change, we also need to calculate its variance and inverse variance to represent the texture features of the whole image more comprehensively:

● Texture variance

Texture variance represents the texture’s period of the defect image. The calculation formula is as follows:

● Texture inverse variance

Texture inverse variance represents the texture changes in the defect image.The larger the value is, the slower the texture changes. The calculation formula is as follows:

In summary, the initial feature set of round steel surface defects we selected in this research is shown in Table 1.

No.19~No.24 represent the texture features of the defect image in one direction. We need to calculate the texture features in 4 directions:${0}^{\circ }，\text{4}{\text{5}}^{\circ }，\text{9}{\text{0}}^{\circ }，\text{13}{\text{5}}^{\circ }$. Therefore, 42 features are selected as the initial feature set of round steel surface defects.

3.   Feature dimensionality reduction (FDR) algorithms

3.1.   Principal Component Analysis

PCA is a multivariate statistical method proposed by Pearson in 1901, and developed by Hotelling in 1993 ^[36]. It is the most widely used FDR algorithm. The main idea of PCA is to map n-dimensional features to k-dimensions, which is a set of orthogonal features also known as the principal components. The process of PCA is to find a set of mutually orthogonal coordinate axes in order from the original space. The first new axis selection of PCA is the direction of the largest variance in the original data, the second new axis selection is the plane orthogonal to the first axis to make the largest variance, and the third axis is the plane orthogonal to the first and second axes to make the largest variance. By analogy, we can get n such axes. From the new coordinate axes obtained in this way, it can be found that most of the variance is contained in the first k coordinate axes, and the variance of the latter is almost 0. Therefore, we can ignore the remaining coordinate axes and only keep the first k coordinate axes with most of the variance. In fact, this is equivalent to only retaining the dimension features that contain most of the variance, while ignoring the feature dimensions that contain almost 0 variance, so as to realize dimensionality reduction of features.

When PCA is used for feature extraction, it should meet the requirements that the variance of samples after dimensionality reduction is as large as possible, the mean square error is as small as possible. PCA looks for the principal axis direction which is used to effectively represent the common features of the same kind of samples ^[37], which is very effective for representing the common features of the same kind of data samples, but it is not suitable to distinguish different sample classes ^[38]. Therefore, PCA is generally not used for feature extraction, but for dimensionality reduction. It needs to be combined with other FDR algorithms to achieve the purpose of feature extraction.

3.2.   Linear Discriminant Analysis (LDA)

LDA is a probabilistic model-based algorithm proposed by Blei et al in 2003 ^[39]. It is a supervised FDR algorithm. Compared with PCA, PCA and LDA have different constraints in the process of projection. The idea of PCA is “the projection method that makes the average points of different categories farthest away”, so its main function is to remove the redundant dimensions of the original data. For data set with different distributions, the maximum variance target after PCA projection will result in data sample mixed or no longer linearly separable. However, the basic idea of LDA is to select an optimal projection direction, so that the data of the same category after projection is closely, and the data of different categories are as far away from each other as possible. The goal of LDA is to reduce the dimensionality of labeled data and project it into a low dimensional space, which meets three conditions at the same time: keep as much information as possible (that is, select the largest feature value as the direction represented by the corresponding feature vector), find the best projection direction that makes samples as easy to distinguish as possible, makes the same type of samples as close as possible and different types of samples as far as possible after projection.

As PCA has advantages in dimensionality reduction, LDA has better distinguishing ability, combine these two algorithms can effectively improving the ability of feature extraction ^[40].

3.3.   Kernel Principal Component Analysis (KPCA)

Many machine learning algorithms assume that the input data is linearly separable. However, in real world, we may face non-linear problems in most cases ^[41]. In this case, PCA and LDA are not the optimal algorithms. KPCA can realize the non-linear dimensionality reduction of data, which is used to process the non-linear separable data set ^[42]. The basic idea of KPCA is: for the matrix X in the input space, we first map all the samples in X to a high-dimensional or even infinite dimensional feature space with a non-linear mapping to make it linearly separable, and then carry out PCA in this high-dimensional space.

Compared with PCA, KPCA can extract the information of the original data to the maximum extent, but it also has the same problem as PCA, that is, it is not suitable to distinguish different sample classes ^[43]. Therefore, KPCA and LDA can also be combined to better extract the features.

3.4.   Auto-encoder (AE)

Deep learning is a feature learning method ^[44]. With the increase of network level, the more complex features are easier to learn, so it is suitable for processing the data with high dimensions. However, too many layers of network will lead to long training time and easy to over-fit. Auto-encoder (AE) is the basic model in deep learning and it can also be used for feature extraction ^[45]. The standard AE is a typical three-layer neural network unsupervised learning model, which includes an input layer, a hidden layer and an output layer, in which the input layer and the output layer have the same dimensions ^[46]. AE can learn the implicit features of input data, which is called “coding”, at the same time, it can reconstruct the original input data with new features, which is called “decoding”. Compared with traditional feature extraction algorithm, AE can better extract the features of original data. The expression of AE is as follows:

In the training process, a loss function is designed to make the input and output of the encoder as similar as possible, that is:

where ${x'} = g\left({f(x)} \right)$.

In general, the mean square error can be used to measure the similarity between the input data and the output data, and its expression is as follows:

where ${E_{x\sim{p_{data}}}}$ indicates that sample x meets the expectation of sample distribution p.

In the learning process, the mean square error may become very small, which will lead to over-fitting. The expectation of AE is to have coding ability for the same type of data, that is to say, the network is required to have strong generalization ability. In this case, the L1 regularization or the Kullback-Leibler divergence (KLD) penalty item needs to be added to the loss function to make AE learn the sparse feature, that is sparse AE. The expression of two loss functions are as follows:

where λ is the regular parameter, which is used to balance loss function and regular term in penalty regression. h_i is the excitation value of the ith neuron.

where β is the weight of control sparsity penalty factor. ρ is the sparsity parameter, generally, its value is set to be close to 0.${\hat \rho _j}$ is the average excitation value of neuron j, which is calculated on the training set. If the value of ${\hat \rho _j}$ is close to 0, only a small number of samples can make this neuron work, which is called the sparse restriction. Since most of the hidden neurons are expected to be “inactive” to learn the specific structures, ${\hat \rho _j}$ is expected to be close to ρ.

When the activation function of hidden layer is Sigmoid, the output value of hidden layer is between (0, 1), KLD penalty item can be used. When the activation function of hidden layer is ReLU, the output value of hidden layer is between (0, $+ \infty$), the L1 regulation function can be used.

In practical application, we can stack multiple sparse AE to obtain a better feature extraction result, which is called the stacked AE. However, the multi-layer network structure will increase the complexity of network training.

4.   Feature extraction and classification based on improved sparse AE

We choose 6 kinds of typical round steel surface defect sample images collected in the actual production line as training set, and each defect contains 500 images. All the 3000 images are preprocessed and the defect parts are segmented. In Section 2.2, 42 features are selected as the initial feature set of round steel surface defects. On this basis, the horizontal direction of the data matrix is 3000 defect images and the vertical direction of the data matrix is 42 feature parameters. In our proposed algorithms, the feature matrix of $\text{3000}\text{×}\text{42}$ is operated as follows (Figure 1):

An architecture overview of the network of purposed algorithms is shown in Figure 2. The shaded part represents the neurons that are not activated in sparse AE. The sparse AE is pre-trained, then as the input of the Softmax classifier.

The Softmax classifier is derived from statistical Logistic regression. The core idea of Logistic regression is to use the logical regression method in classification, which can judge the input data and then output a single discrete result to classify the extracted features. It first determines the probability of the extracted features and then classifies them. For the dataset, it gives the probability sum of all vectors to be 1. In Softmax, the specific mapping probability given by the function makes the sum of the probabilities of all categories 1. The Softmax functions are as follows:

Where z is the vector of inputs, the results are mapped from the exponential domain to the probability, and the sum of probabilities is guaranteed to be 1. The Softmax loss function mainly uses the cross-entropy loss function, and its formula is as follows:

For the Softmax loss function, it can be understood from the point of view of information theory, and the loss function can be regarded as the entropy of two probabilities, and its formula is as follows:

The purpose of the loss function is to measure the error between the true classification result and the predicted classification result, and then optimize and modify it based on this value. Where p represents the probability of true classification, and q represents the probability of predicting classification.

4.1.   PKAE

PCA is a linear FDR algorithm and KPCA is a nonlinear FDR algorithm. If PCA or KPCA is used for feature extraction, the feature extracted is the linear or nonlinear representation of the original input data. However, the relationship between round steel surface defects is complex. There may be not only linear relationships, but also non-linear relationships. It is necessary to combine PCA and KPCA features to obtain more complex features, then as the input of sparse AE to extract optimal features. In order to verify the performance of the Softmax classifier, k-fold cross validation is adopted. The original training set is divided into six parts, one of which is taken as the test set each time, and the other five parts are taken as the training set. Iterating 20 times, and taking the mean value of classification accuracy as the final experimental results.

Suppose that the sample matrix is X ($X = {\left({{x_{ij}}} \right)_{n \times m}}$). Where n is the total number of samples, m is the number of features and ${x_{ij}}$ represents the jth feature value of the ith sample, where $i = 1, 2, \ldots n$, $j = 1, 2, \ldots, m$. The specific steps of PKAE are as follows:

4.1.1.   Feature extracted by PCA

● Step 1

Standardize the original data, all the samples are subtracted from the mean value of corresponding feature. The calculation formula of the mean value is as follows:

● Step 2

Calculate the covariance matrix P ($P = {\left({{r_{jk}}} \right)_{m \times m}}$), where m is the number of features. ${r_{jk}}$ represents the correlation between the jth and kth feature, where $j = 1, 2, \ldots, m$, $k = 1, 2, \ldots, m$:

● Step 3

Calculate the eigenvalue λ_i and the eigenvector е_i of the covariance matrix P:

● Step 4

Record the resulting eigenvalues in the order of large to small:${\lambda _1} \geqslant {\lambda _2} \geqslant \cdots \geqslant {\lambda _k}$, calculate the contribution rate of each principal component. The formula of contribution rate is as follows:

where $g = 1, 2, \cdots, k$. The higher the contribution rate is, the stronger the information of the original variables contained in the principal component is. Generally, the top eigenvalues with cumulative variance contribution is more than 85% are considered.

● Step 5

Transform the original sample matrix X into a new matrix Y₁ (${Y_{\rm{1}}} = {\left({{Y_{ij}}} \right)_{n \times {m_1}}}$), where $i = 1, 2, \ldots, n$, $j = 1, 2, \ldots, {m_1}$:

where $\left[{{e_1}, {e_2}, \cdots {e_{{m_{\rm{1}}}}}} \right]$ represents a new feature space composed of m1 feature vectors. m1 are the principal components extracted by PCA.

4.1.2.   Feature extracted by KPCA

● Step 1

Mapping each original sample data ${x_i}$ into high dimensional space by nonlinear function $\Phi \left(x \right)$:

where $i = 1, 2, \ldots, n$.

● Step 2

Calculated the kernel matrix ${\left[K \right]_{ij}} = {K_{ij}} = \left({\Phi \left({{x_i}} \right), \Phi \left({{x_j}} \right)} \right)$. Use Gaussian function as the kernel function, then the expression function of K is as follows:

where δ is the width, given by experience.

● Step 3

Calculate the central kernel matrix:

● Step 4

Calculate the eigenvalue and eigenvector of $\tilde K$

● Step 5

Record the resulting eigenvalues in order of large to small:${\lambda _1} \geqslant {\lambda _2} \geqslant \cdots \geqslant {\lambda _k}$, calculate the contribution rate of each principal component. Generally, the top eigenvalues with cumulative variance contribution is more than 95% are considered.

● Step 6

Transform the original sample matrix X into a new matrix Y₂ (${Y_{\rm{2}}} = {\left({{Y_{ij}}} \right)_{n \times {m_{\rm{2}}}}}$), where $i = 1, 2, \ldots, n$, $j = 1, 2, \ldots, {m_{\rm{2}}}$:

4.1.3.   Feature extracted by PKAE

● Step 1

Add KPCA features belonging to the same sample to PCA features to get a new sample matrix Y ($Y = \left[{{Y_1}, {Y_2}} \right]$):

● Step 2

The new sample matrix is divided in to training set and test set by 6-fold cross validation. Use sparse AE to learn the optimal features. Sigmoid is adopted as the activation function of hidden layer, in this case, the equation (36) is used as the loss function of sparse AE.

● Step 3

Add Softmax classifier at the top layer of the network, use the labeled data to train Softmax, and use back-propagation (BP) algorithm to adjust the weights in the network. The Softmax loss is used as the loss function.

● Step 4

Train the whole network and conduct classification by Softmax.

4.2.   PLAE

PCA and LDA are both linear FDR algorithms, combine the features by these two FDR algorithms, then as the input of sparse AE, can learn the linear relationship and the possible hidden nonlinear relationship between round steel defects. In order to verify the performance of the Softmax classifier, k-fold cross validation is adopted. The original training set is divided into six parts, one of which is taken as the test set each time, and the other five parts are taken as the training set. Iterating 20 times, and taking the mean value of classification accuracy as the final experimental results.

Suppose that the sample matrix is X ($X = {\left({{x_{ij}}} \right)_{n \times m}}$). n is the total number of samples, m is the number of features, and ${x_{ij}}$ represents the jth feature value of the ith sample, where $i = 1, 2, \ldots n$, $j = 1, 2, \ldots, m$. The specific steps of PLAE are as follows:

4.2.1.   Feature extracted by PCA

This step is the same as Section 4.1, after feature extracted by PCA, the original sample matrix X is transformed into a new matrix Y₁.

4.2.2.   Feature extracted by LDA

LDA uses the following Fisher criteria to obtain the projection vector:

where ${S_b}$ represents the inter-class dispersion matrix, and ${S_w}$ represents the intra-class dispersion matrix. The calculation formulas are as follows:

where ${u_i} = \frac{1}{{{n_i}}}\sum\nolimits_{j = 1}^{{n_i}} {x_j^i}$ represents the average value of the features of all samples belonging to class i, and $u = \frac{1}{n}\sum\nolimits_{i = 1}^c {\sum\nolimits_{j = 1}^{{n_i}} {x_j^i} }$ represents the average value of the features of all samples. c represents there are a total of c defect types.

The optimal projection vector α is the eigenvector corresponding to the maximum eigenvalue of the equation: ${S_w}\alpha = \lambda {S_b}\alpha$. Let $\left[ {{\alpha }_{\text{1}}},{{\alpha }_{\text{2}}}\ldots {{\alpha }_{k}} \right]\in {{R}^{m\times k}}$ be the set of the first k eigenvectors selected, where k = c-1. Then we can transform the original sample matrix X into a new matrix Y₂ (${Y_{\rm{2}}} = {\left({{y_{ij}}} \right)_{n \times k}}$), where $i = 1, 2, \ldots n$, $j = 1, 2, \ldots, k$:

4.2.3.   Feature extracted by PLAE

● Step 1

Add LDA features belonging to the same sample to PCA features to get a new sample matrix: $Y = \left[{{Y_1}, {Y_2}} \right]$

● Step 2

The new sample matrix is divided in to training set and test set by 6-fold cross validation. Use sparse AE to learn the optimal features. Sigmoid is adopted as the activation function of hidden layer, in this case, the equation (36) is used as the loss function of sparse AE.

● Step 3

Add Softmax classifier at the top layer of the network, use the labeled data to train Softmax, and use back-propagation (BP) algorithm to adjust the weights in the network. The Softmax loss is used as the loss function.

● Step 4

Train the whole network and conduct classification by Softmax.

4.3.   KLAE

KPCA is a nonlinear FDR algorithm. The feature extracted by LDA is the linear representation of the original data. Combining these two FDR algorithms can extract the linear and nonlinear relationship of the round steel surface defects, then as the input of sparse AE to extract optimal features. In order to verify the performance of the Softmax classifier, k-fold cross validation is adopted. The original training set is divided into six parts, one of which is taken as the test set each time, and the other five parts are taken as the training set. Iterating 20 times, and taking the mean value of classification accuracy as the final experimental results.

Suppose that the sample matrix is X ($X = {\left({{x_{ij}}} \right)_{n \times m}}$). n is the total number of samples, m is the number of features, and ${x_{ij}}$ represents the jth feature value of the ith sample, where $i = 1, 2, \ldots n$, $j = 1, 2, \ldots, m$. The specific steps of KLAE are as follows:

4.3.1.   Feature extracted by KPCA

This step is the same as Section 4.1, after feature extracted by KPCA, the original sample matrix X is transformed into a new matrix Y₁.

4.3.2.   Feature extracted by LDA

This step is the same as Section 4.2, after feature extracted by LDA, the original sample matrix X is transformed into a new matrix Y₂.

4.3.3.   Feature extracted by KLAE

● Step 1

Add LDA features belonging to the same sample to KPCA features to get a new sample matrix: $Y = \left[{{Y_1}, {Y_2}} \right]$

● Step 2

The new sample matrix is divided in to training set and test set by 6-fold cross validation. Use sparse AE to learn the optimal features. Sigmoid is adopted as the activation function of hidden layer, in this case, the equation (36) is used as the loss function of sparse AE.

● Step 3

Add Softmax classifier at the top layer of the network, use the labeled data to train Softmax, and use back-propagation (BP) algorithm to adjust the weights in the network. The Softmax loss is used as the loss function.

● Step 4

Train the whole network and conduct classification by Softmax.

5.   Experimental results and discussions

The data samples used in the experiment are 3000 selected round steel surface defect images as introduced in Section 4, which include 6 kinds of defects and 500 images of each defect. The images are preprocessed and defect parts are segmented. In order to verify the advantage of our proposed algorithms, we compared proposed algorithms with individual sparse AE and individual traditional FDR algorithm in classification accuracy. At the same time, we also provided a comparison of network training time between the proposed algorithms and individual sparse AE. Through the comparisons, the algorithm with best classification accuracy and minimum network training time are selected as the optimal algorithm. Finally, the feasibility and effectiveness of the optimal algorithm are verified in the context of actual production, 4 batches of round steel surface images collected on an actual production line were used as the test set to compare the defects classification accuracy between optimal algorithm and individual traditional FDR algorithm.

5.1.   Experimental environment and parameter settings

The experimental environment was win 7, 32-bit system, i7 processor, 3.4 GHz main frequency and 4GB memory. All the programmings were completed on Matlab 2014. In order to obtain more advanced and complex features, the Sigmoid was selected as the active function. At the same time, the number of network training will affect the classification accuracy. After several experiments, the number of network training was selected as 1500, and the learning rate was set as 0.5. In the actual production line, the detection of surface defects, the change of production process parameters, the adjustment of energy data and the diagnosis of equipment operation parameters all share an ERP system, in this case, each function unit is required not to occupy too much computing resources. Therefore, the lower the network complexity is and the less network training time is, the better.

5.2.   Results and discussions

In order to retain as much information as possible, in this research, the cumulative principal component contribution rate of PCA and KPCA was selected as 99%. The kernel function of KPCA is Gaussian kernel function. As a result, 15 features were remained after PCA, 12 features were remained after KPCA. The features remained after LDA were the number of defect types minus 1 (that is 5).

The number of hidden layer nodes will affect the performance of final network. At present, the common method to determine the number of hidden layer nodes is trail and error method. The same sample set is used to train the network with different numbers of hidden layer nodes, and the value corresponding to the minimum network error is selected. If the number of nodes in the input layer is n and the number of nodes in the output layer is l, the determination of the initial number of nodes in the hidden layer m is generally calculated by the following empirical formulas:

where $\alpha$ is a constant between 1 and 10.

The advantage of sparse AE is that when the number of nodes in the hidden layer is large, it can learn the advanced features of input data by adding sparsity constraints. Therefore, formula (56) was adopted to determine the initial number of nodes in the hidden layer in this paper, then increased the number of nodes in hidden layer for training and compared the error of network with different numbers of nodes in hidden layer. The results are as follows (from Table 2 to Table 4):

It can be seen from Table 2 to Table 4 that, in most cases, with the increase of the number of nodes in the hidden layer, the output error of the network decreases slightly, but not obviously. At the same time, when the number of nodes in the hidden layer is equal to the input layer and output layer, the error value is far less than the error requirement of 0.001. Therefore, in order to reduce the network complexity and reduce the network training time, the number of nodes in the hidden layer was set to be the same as the input layer and the output layer. In this case, in PKAE, the node number of input layer, hidden layer and output layer were the same, which was the sum of dimensions of PCA features and KPCA features (that is 27). In PLAE, the node number of input layer, hidden layer and output layer were the same, which was the sum of dimensions of PCA features and LDA features (that is 20). In KLAE, the node number of input layer, hidden layer and output layer were the same, which was the sum of dimensions of KPCA features and LDA features (that is 17). The experimental results of defects classification accuracy are shown in Table 5. For the better comparison, Softmax classifier is used for defects classification by all algorithms.

It can be seen from Table 5:

(1) The classification accuracy of the proposed algorithms (PKAE, PLAE and KLAE) are significantly better than individual traditional FDR algorithm and sparse AE.

(2) The classification accuracy of PKAE is worse than PLAE and KLAE. This may be due to the similarity between different types of round steel surface defects, and LDA has the advantage in distinguishing different defect types.

(3) The classification accuracy of KLAE is better than PLAE. It is proved that there are both linear and nonlinear relationships between round steel surface defects.

(4) KLAE has the best classification accuracy, and the classification accuracy on scratches reaches 99.12%. However, the classification accuracy of fold over and cracks are relatively low, which may be due to the similarity between these two defect types in shape and grayscale features, and sometimes it is difficult to distinguish them in the image.

We also compared the network training time of the proposed algorithms with individual sparse AE. The experimental results of network training time are shown in Table 6.

It can be seen from Table 6 that compared with individual sparse AE, the network training time of the proposed algorithms is greatly reduced. Among of them, KLAE requires less network training time, which is 45.99s. As KLAE has advantages in both classification accuracy and network training time, we finally chose KLAE as the final feature learning network.

The experimental results of defects classification sensitivity and specificity were also provided to prove the effectiveness of KLAE. The results are shown in the table below (Table 7).

It can be seen from the Table 7 that KLAE has both high defects classification sensitivity and specificity, so the algorithm proposed in this paper is very effective.

In order to further verify the feasibility of KLAE in the practical industrial application, 4 batches of round steel surface images collected on the actual production line were used as the test set to compare the defects classification accuracy of KLAE and traditional FDR algorithm. The defects classification process of KLAE is shown in Figure 3:

After image preprocessing and defect image screening, the number of defect images in each batch is: 401,428,500,379. Then the features of defect part were extracted by KLAE, and classified by Softmax. The experimental results are shown in Table 8.

As can be seen from Table 8 compared with traditional FDR algorithm, KLAE can significantly improves round steel surface defects classification accuracy in the actual production line. In actual production process, there may be multiple defects coverage in one round steel surface image, which may affect the accuracy of classification. In general, the defects classification accuracy in Table 5 can satisfy the actual production application requirements. In this case, the proposed algorithm is feasible and effective.

6.   Conclusion

Surface defects defection is particularly important for enterprises to meet different production standards and requirements. Normally, in defect feature extraction, we first obtain the defect area of the defect image by image preprocessing and defect segmentation, select the original feature set of defects by prior knowledge, evaluate the importance of selected features by using the traditional FDR algorithms and extract the optimal features according to the importance rank of features. In this paper, a feature extraction and classification algorithm based on improved sparse AE is proposed. We adopt three traditional FDR algorithms at the same time, combine the defect features obtained in pairs, take merged defect features as the input of sparse AE, then use the “bottleneck” of sparse AE to conduct defects classification. The experimental results show that our proposed algorithm KLAE can extract the optimal features of round steel surface defects with less network training time than individual AE, and finally get higher classification accuracy than individual traditional FDR algorithm in the actual production line.

Due to the limitation of time and conditions, this research still has some limitations. In the future research, improvements can be made in the following aspects:

(1) Parameters in the network, such as the number of nodes in the hidden layer, the number of network training times and the activation function are all given based on experience, which will affect the classification accuracy of the final network.

(2) Considering that stacked AE may increase the complexity of network training, this research only use the basic sparse AE with one hidden layer, which is proved can satisfy the actual production application requirements. However, stacked AE is also valuable for study to obtain better defect classification accuracy.

(3) With the increase of production, new types of defects will also appear. In this research, we have considered the 6 typical round steel surface defects. In the future research, more abundant defects information can be obtained by updating the defect sample database to better classify the defects in the actual production line.

Funding

This research was funded by the Natural Science Foundation of China (NSFC) under Grant no. 51435009, Science and technology innovation 2030 - a major project of new generation artificial intelligence: virtual reality integration and adaptive production scheduling of complex manufacturing under Grant no. 2018AAA0101704 and supported by “Program for HUST Academic Frontier Youth Team under Grant no. 2017QYTD04.”

Conflict of interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.