Predicting the weld width from high-speed successive images of the weld zone using different machine learning algorithms during laser welding

Nomenclature and Abbreviations: BPNN: Back Propagation Neural Network; SVR: Support Vector Regression; RBFNN: Radial Based Function Neural Network; SVM: Support Vector Machine; PCA: Principal Component Analysis; PCs: Principal Components; ICI: Inline Coherent Imaging; MAE: Mean Absolute Error; RMSE: Root Mean Square Error; ED: Experiment Data; PD: Predicted Data; PE: Prediction Error; GPA: Global Prediction Accuracy; LPA: Local Prediction Accuracy

1.   Introduction

Laser welding obtains increasingly widespread application in the automotive industry, shipbuilding industry, aerospace, and microelectronics industry ^[1,2,3,4,5], due to its prominent merits such as high density of laser power, high productivity in industry, high penetration of weld seam, and the process parameters are easy to be controlled. External magnetic field assisted laser welding can ameliorate the weld quality to some extent by regulating the plasma, changing the mass transfer and heat transfer process of the weld zone ^[6,7]. The fluctuation of keyhole and molten pool still exist, this instability may cause the uneven penetration of weld seam and weld defects like pores and spatters ^[8,9]. To guarantee the quality of weld seam, it is an effective way to monitor keyhole ^[10] and molten pool ^[11] on-line with advanced sensor system to reflect weld quality by predicting the weld penetration, weld width or weld defects. This monitoring technique can be mainly divided into two important parts.

On the one hand, the intense laser-induced plasma ^[12] and metal vapor flames ^[13] cause the keyhole and molten pool are difficult to be monitored on-line to capture clear useful images. Consequently, A great quantity of exploration work has been done by many scientific researchers in the selection of visual monitoring equipment and the construction of the monitoring platform. Fan et al. ^[14] devised a three-optical-route visual sensor which can monitor the molten pool synchronously from three directions. Useful image sequences were recorded for extracting data. Fabbro et al. ^[15] observed the dynamic changes of keyhole by applying a coaxial camera and a paraxial photodiode. The top surface of keyhole was captured with ideal precision, but the inside morphology was hard to obtain. Kaierle et al. ^[16] researched coaxial monitoring system with charge-coupled device or complementary metal oxide semiconductor camera, and in ^[17], a near-infrared camera was added to this system to acquire the temperature distribution of welding zone. In recent years, the methods for obtaining internal geometry information of keyhole (such as the depth of keyhole) have been gradually increased ^[18,19]. Webster et al. ^[20,21] applied the inline coherent imaging (ICI) technique to obtain keyhole depth. The ICI metrical beam coincides with laser beam but does not interact with each other. ICI also owns the advantage like high speed and high dynamic range, but the expensive equipment may hinder its application in industry. In this paper, an on-line vision monitoring system based on high-speed camera is built to obtain keyhole and molten pool images. Results showed that the obtained data can be used to predict the weld quality after image processing.

On the other hand, it is essential to choose efficient algorithms to predict weld penetration, weld width or weld defects by analyzing the monitoring data. The application of machine learning algorithms in the establishment of weld quality forecasting model is more and more widespread ^[22,23,24]. Among them, the back propagation neural network (BPNN), radial basis function neural network (RBFNN) and support vector machine (SVM) are commonly used algorithms, and acquired reliable and credible results. Gao et al. ^[25] applied magneto optical imaging technique which based on the magnetic induction principle and Faraday rotation effect ^[26] to monitor welding process, and BPNN was used to diagnose weld defects. Results achieved a degree of accuracy with 93.5%. Wang et al. ^[27] used two machine learning algorithms (i.e. multiple linear regression and BPNN) to evaluate the weld quality by analyzing the monitoring data. Results showed that BPNN had a better performance. System error and measurement error are the common problem in monitoring and these errors also have the characteristics of non-linear and unknown distribution. Luo et al. ^[28] predicted the penetration of weld seam and inclination angle with RBFNN system in the transient state when process parameters changed abruptly. SVM is kernel based machine learning algorithm which is used in defects classification works and regression problems (i.e. support vector regression (SVR) model). The morphological geometry of metal vapor plume is related to the stability of welding process. Wang et al. ^[29,30] used improved SVM model to classify these dynamic features and the classification accuracy was higher than 90%. In ^[31], the established predicting model based on SVM and cross validation method also successfully classified weld defects (porosity) with high accuracy. The BPNN, RBFNN and SVM models all have their own advantages in laser welding monitoring. Choosing the suitable algorithm can reduce a lot of prophase work and increase prediction accuracy, above three basic machine learning algorithms were compared in this paper, aim to provides a guideline for the selection of the algorithm.

In this paper, an on-line vision system is established to continuously capture the keyhole and molten pool images. The welding zone is illuminated by the pulsed laser-assisted illumination equipment. After image processing and feature extraction, six keyhole features and one molten pool feature are extracted from the captured images. To improve the modeling efficiency, the principal component analysis (PCA) algorithm is applied to reduce the dimension of the extracted data. Then, different number of principal components (PCs) are chosen as the input data of these three models. Weld width is an important parameter for evaluating weld quality which will be used as the output data. Finally, the BPNN, RBFNN and SVR models are established and the prediction performances of these models are compared. With the help of these advanced algorithms, the weld quality can be monitored in real time. This work aims to find suitable methods to predict the weld width by comparing different PCs and machine learning algorithms. The results show the effectiveness of the applied algorithms and this work can provide a guideline for the selection of machine learning algorithm during laser welding on-line monitoring.

The remainder of this paper is organized as follows. In Section 2, the experimental details of the external magnetic field assisted laser welding on the AISI 2205 duplex stainless steel plates are given. The description of the image process and feature extraction of the keyhole and molten pool is presented in Section 3. In Section 4, three different machine learning algorithms are compared under different number of PCs. In section 5, the results and discussion about BPNN, SVR and RBFNN models are given. Finally, the concluding remarks are given in Section 6.

2.   Experiment setup

Figure 1 demonstrates the equipments of laser welding on-line monitoring. An ABB 6-axis robot (IRB 4400 M 2004), and a ytterbium-doped fiber laser device (IPG YLR-4000, 4 kW), a worktable that can supply different magnetic field by changing the current, and an argon shielding gas equipment (flow rate: 25 L/min) are assembled as the welding system. The AISI 2205 duplex stainless steel plates are machined to the appropriate size (i.e. 150 × 100 × 4 mm³) for welding. Table 1 shows the chemical composition in weight percent of the base metal. The diameter of the light spot on workpiece is about 300 μm by utilizing a focusing lens which is installed in the laser head and the focus length is 250 mm. A phantom V611 high-speed camera (Fps: 5000 frame/s, 500 pixels × 256 pixels) and an auxiliary pulsed diode laser light illuminant system (wavelength: 808nm, 400W) are equipped at the side of the worktable to photograph the keyhole and molten pool. A narrow-band filter is installed in front of the high-speed camera to filter the radiations from plasma, spatters radiation and laser beam etc..

Figure 1.  The schematic diagram of the experimental setup.

DownLoad: Full-Size Img PowerPoint

Table 1.  Chemical composition of AISI 2205 duplex stainless steel (wt.%) ^[32].

Elements	C	Ni	Mo	Cr	Mn	Si	Fe
Composition	0.025	5.53	3.03	22.73	1.89	0.34	Balance

---

 | Show Table

DownLoad: CSV

Two experiments results are compared in this paper. The welding speed is 1m/min. The laser power is 3 kW and magnetizing current are 10 A and 20 A. The high-speed camera takes images at the speed of 5000 f/s. 5500 consecutive images in two experiments are selected at the same section on two weld seams. The length of the weld seam is divided into the same amount as the captured images. The information of each image corresponds to each weld width. Two weld seams are scanned to high-resolution (4898pixels × 803pixels) images for obtaining the weld width. The upper edge and the lower edge of the weld seam are extracted by applying image processing method. The weld width could be obtained by calculating the distance between the upper and the lower edges of the weld seam. Figure 2 shows the detailed implementation procedures in MATLAB 2016a. The definition features were extracted and analyzed with PCA algorithm, and the corresponding weld width data were also obtained. Finally, the obtained data were applied to train and test the BPNN, RBFNN and SVR models.

Figure 2.  The flowchart of the implementation procedure in MATLAB 2016a.

DownLoad: Full-Size Img PowerPoint

3.   Feature extraction and analysis

3.1.   The definition of features

In this section, the method for obtaining features of the keyhole and molten pool is introduced in detail. Seven features are defined and the schematic of these features are shown in Figure 3. The width of keyhole (${W_k}$), the length of keyhole (${L_k}$), the area of keyhole (${A_k}$), the perimeter of keyhole (${p_k}$), the center coordinates of keyhole (${X_k}$, ${Y_k}$) and the width of molten pool (${W_m}$) are defined according to the appearance of the keyhole and molten pool.

Figure 3.  The definition of the extracted features.

DownLoad: Full-Size Img PowerPoint

3.2.   Image processing

Figure 4 shows the detailed information of the extraction process of the keyhole features. The first step is to take out the keyhole part from the original image to remove a large amount of useless information and reduce the influence of the surrounding spatters and other noises, so the information of the keyhole will be more accurate. Then, a linear grayscale transformation algorithm is applied to the intercepted image to linearly stretch treat each pixel of the image, aim to enhance the contrast between the keyhole and the surroundings. Next, the median filtering technology is used to eliminate noise interferences such as outliers and line segments. Threshold segmentation operating converts the filtered image into the binary image so that the keyhole will be composed of white pixels and the surroundings will be filled with black pixels. Finally, the keyhole morphological data can be obtained by counting the number of white pixels.

Figure 4.  The keyhole features extraction processing.

DownLoad: Full-Size Img PowerPoint

As shown in Figure 3, the molten pool is symmetric about the center coordinates (${X_k}$, ${Y_k}$) of the keyhole. Consequently, the molten pool can be divided into two equal parts. Half width of molten pool can be obtained by image processing of clearly illuminated part by auxiliary light, so the whole weld width will be acquired. The molten pool image processing steps are the same with keyhole image processing. The detailed information is shown in Figure 5.

Figure 5.  The molten pool feature extraction processing.

DownLoad: Full-Size Img PowerPoint

Figure 6 and Figure 7 exhibit the curves of seven features extracted from 5500 images. It can be seen that the features extracted from different experiments (10 A and 20 A) have the same tendency. The keyhole features are stable with few deviated points except the ${X_k}$. The value of ${X_k}$ become smaller during laser welding, indicating that the weld seam deviates from the default straight line. The curve about the width of molten pool fluctuates sharply which means the molten pool is very unstable during laser welding.

Figure 6.  The extract data through the image processing (10A).

DownLoad: Full-Size Img PowerPoint

Figure 7.  The extract data through the image processing (20A).

DownLoad: Full-Size Img PowerPoint

3.3.   Analysis the features with PCA

Reducing the dimension of data helps to increase modeling efficiency. In this paper, the PCA algorithm ^[33] is used to reduce the dimensions of the input data. In this paper, the feature vector ${N_i}$ consists 7 features and every feature combined with 5500 samples. In these samples, ${N_{\max }}$ and ${N_{\min }}$ are the maximal and minimal values. The feature vector N was normalized between –1 and 1 by the formula below,

Then the PCA algorithm will transform ${\hat N_i}$ into a new vector ${M_i}$ with the Eq 2,

where $L$ is the 7 × 7 orthogonal matrix. The $t$th eigenvector of the $I$ is the $t$th column ${x_t}$. By using the Eq 4, the eigen value problem can be solved.

In Eq 4, ${\lambda _t}$ is one of the eigen value of $I$ and ${x_t}$ is the corresponding eigenvector. The component ${M_i}$ can be calculated as the orthogonal transformations of ${\hat N_i}$, i.e.,

This method can convert a set of potentially related variables into a set of independent linear variables which were called the PCs. Different number of PCs will be selected as the inputs of three models which will be introduced in Section 4. The process of dimensional reduction with PCA algorithm is shown in the Figure 8. There are eight important steps during the PCA process.

Figure 8.  The process of the PCA algorithm.

DownLoad: Full-Size Img PowerPoint

Seven PCs are obtained and variance of each PC are shown in Figure 9. In order to get the appropriate dimension after PCA processing as the input data. The different number of PCs are chosen train the models for comparing their performance, including 2 PCs, 3 PCs, 4 PCs, 5 PCs. The prediction accuracy of models could be improved as much as possible with accurately input PCs.

Figure 9.  The results of the PCA algorithm with seven extracted features in two experiments.

DownLoad: Full-Size Img PowerPoint

4.   The introduction of BPNN, RBFNN, SVR models

4.1.   The BPNN model

The network structure of BPNN is a forward multilayer networks ^[34]. The learning process consists of two steps: namely positive propagation of signals and backward propagation of errors. The input samples are processed by the hidden layer (one or more), and the results will be delivered to the output layer. If the actual output of the output layer does not match the expected outcome, then the reverse propagation phase of the error would be reversed.

During the initialized phase, the number of neurons in input layer, hidden layer, and output layer are set as $n$, $m$ and $l$. The threshold of hidden layer and output layer are set as $\alpha$ and $\beta$. The $f$ is the active functions of hidden layer.

The output ${H_j}$ of neuron $j$ in hidden layer can be calculated by the fellow formula,

where ${w_{ij}}$ indicates the connection weigh between the input layer neuron $i$ and the hidden layer neuron$j$, ${x_i}$ is the input variable and ${\alpha _j}$ denotes the threshold of neuron $k$ in the output layer.

Then the result ${Q_k}$ of the output layer would be obtained by Eq 7,

where ${Y_k}$ is the desired outcome, so the predicted error ${E_k}$ can be calculated by Eq 8,

The smaller the predicted error ${E_k}$, the better predicted performance. The adjusted weight will change the output. The weight ${w_{ij}}$ connects the input layer and the hidden layer will be updated by Eq 9. The weight ${w_{jk}}$ connects the hidden layer and output layer will be replaced by Eq 10. Thresholds of hidden layer and output layer are renovated by Eq 11 and Eq 12, respectively.

For each sample, if the error is smaller than the threshold or reaching the maximum iterations, the training process will be finished. Otherwise the training process will go back to continue the above procedures again.

4.2.   The RBFNN model

The RBFNN ^[35] model uses discrete multivariate data to fit unknown functions. The core of this model is to use the functionto approximate the real variable $F\left(x \right)$. The function $f\left(x \right)$ adopted $\psi \left(r \right)$ as the basis function which determined by the radial distance $r$ from each data point ${x_i}$. The basic form of radial basis function is,

Then the simulation function can be expressed as,

where the distance ${r_i}\left(x \right) = \left\| {x - {x_i}} \right\|$ is the $i$th sampling point $x$ in the design space ${x_i}$. $\psi \left(r \right)$ is the basis function. is the European-style norm. $a$ is the non-negative constant and $\alpha$ is the weighting coefficient of the radial basis function $\psi \left(r \right)$.

When using the above equation as a predicted model, the following interpolation conditions must be satisfied,

Then the equations are got as below,

When the sample points do not coincide, there is a unique solution when $\psi \left(r \right)$ is the positive definite function, give the result as Eq 17.

The typical RBFNN structure consisted of an input layer, a hidden layer, and an output layer. It reflects the functional relationship $f:x \to \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{y}$ between multiple inputs $x \in {R^n}$ and multiple outputs $y \in {R^n}$ of the system. The input node passes the signal to the hidden layer, and the output node is usually a simple linear function.

4.3.   The SVR model

The SVR model ^[36] is based on the application of SVM model to regression problems ^[37]. SVR has excellent non-linear processing capability, especially when dealing with high-dimensional data. Each sample point would be mapped to a high-dimensional feature space by using a nonlinear function. Then performing linear regression in the high-dimensional feature space to achieve the effect of linear regression in the original space. The function$f$can be expressed as blew,

where $w$ is the weight vector and the $a$ is constant. To reduce the risk caused by $w$, the structural risk functions in Eq 19 is introduced,

According to the structural risk criterion $\varepsilon$ in the statistics as the loss function, the minimization objective function in Eq 20 is used to obtain $w$ and $a$.

where $C$ is the penalty parameter. ${\xi _i}$ and $\xi _i^ *$ denote the introduced slack variables. $\delta$ is the pre-set value and ${y_i}$, ${y_i} \in \left\{ { - 1, 1} \right\}$ is the mappings of sample space. Therefore, the minimum solution of the objective function $F\left({w, {\xi _i}, \xi _i^ * } \right)$ is obtained. The output of the SVR model is shown in Eq 21.

where $K\left({x, {x_i}} \right)$ is the kernel function. The $f$ is the output result and $x$ is the multidimensional space vector.

5.   Results and discussion

When the magnetizing current is 10 A, the weld width is shown in Figure 10. From Section 3.3, seven features are extracted from 5, 500 continuously images, the samples of weld width numbered from 1, 001 to 6, 500 are corresponding to those analyzed 5500 images of the keyhole and molten pool. Consequently, 5500 samples of 2 PCs, 3 PCs, 4 PCs and 5 PCs are chosen as the input data of these three models and the corresponding weld width are selected as the output data. The former 5000 samples are chosen as the training set and the last 500 samples are used as testing set to establish the BPNN, RBFNN and SVR models. The ratio of training samples to testing samples is 10:1. To make these three models rapidly converge, all the samples in the training set and testing set are normalized to the interval of [0, 1] in MATLAB 2016a.

Figure 10.  The weld width and the sampling section (10A).

DownLoad: Full-Size Img PowerPoint

To judge which PCs is the best combination of PC to establish three models, and which model owns the best performance. The mean absolute error (MAE) and the root mean square error (RMSE) ^[38,39,40] are used as the accuracy metrics. The MAE is the average of the absolute error which can reflect the actual situation of the prediction error (PE). The local prediction accuracy (LPA) of model will be better if the MAE is smaller. As for RMSE, it can be used to ensure the accuracy index consistent with the unit of output performance. RMSE is more commonly used in the global accuracy test of model. The smaller the RMSE is, the higher the global prediction accuracy (GPA) of the model is.

where ${y_i}\left({{x_i}} \right)$ represents experimental data (ED) and ${\hat y_i}\left({{x_i}} \right)$ is the corresponding predicted data (PD). The ${X_t}$ is the set of verification points, $t$ is the number of verification points and ${x_i}$ is the verification point.

The results of MAE and RMSE of three models with different PCs are shown in Figure 11.

Figure 11.  The MAE and RMSE in different models and different PCs (10A).

DownLoad: Full-Size Img PowerPoint

It can be concluded from Figure 11 that it is better to choose 2 PCs as input data, due to the MAE and RMSE are obvious smaller than other PCs. When 2 PCs are chosen as input data, the PE curves of three models are calculated with 500 ED samples from the testing set and their corresponding PD. The PE curves, MAE and RMSE values of models are shown in Figure 12.

Figure 12.  The PE curve and MAE, RMSE values of different models with 2 PCs as input (10A).

DownLoad: Full-Size Img PowerPoint

The PE curve of BPNN model is smoothest, and BPNN model has the smallest MAE. The RMSE of BPNN model is bigger than SVR model, but the PE curve of the SVR model fluctuates sharply from No. 5300 to No. 5500. Synthetically speaking, BPNN model performs better than SVR model. As for RBFNN model, the PE curve fluctuates acutely and has the biggest MAE and RSME among three models, so the performance of the RBFNN model is worst. All in all, BPNN model performs better than SVR model and SVR model performs worst when the magnetizing current is 10A.

When the magnetizing current changed to 20 A and other conditions are the same with the 10 A experiment. By applying the image processing method, the weld width is extracted and shown in Figure 13.

Figure 13.  The weld width and the sampling section (20 A).

DownLoad: Full-Size Img PowerPoint

In MATLAB 2016a, the same methods are taken to deal with the normalized 5000 training data and 500 testing data. The results of MAE and RMSE of three models with different PCs are shown in Figure 14.

Figure 14.  The MAE and RMSE in different model and different PCs (20A).

DownLoad: Full-Size Img PowerPoint

It can be concluded from Figure 14 that the BPNN model and SVR model have almost similar performance no matter which PCs is chosen. When choose 4 PCs and 5 PCs as input data, RBFNN model performs badly due to big MAE. The 3 PCs owns better RMSE than 2 PCs and almost identical MAE. Comprehensive speaking, it is better to choose 3 PCs as the input data. The PE curves, MAE and RMSE values of models are shown in Figure 15.

Figure 15.  The PE curves and MAE, RMSE values of different models with 3PCs as input (20A).

DownLoad: Full-Size Img PowerPoint

From Figure 15, the MAE and RMSE of BPNN is smaller than SVR model and the PE curve is also smoother. Consequently, BPNN model performs better than SVR model. RBFNN model has the biggest MAE and RMSE, and the PE curve fluctuates sharply. Overall, the performance of BPNN model is better than SVR model, and RBFNN model is worst when compared with BPNN model and SVR model.

It can be concluded from the above two different experiments that BPNN model performs better than SVR model, and RBFNN model performs worst.

6.   Conclusions

In this research, an on-line monitoring system is established to observe the dynamic behavior of the keyhole and molten pool and the prediction performance of three machine learning algorithms are compared with different PCs as inputs in two different experiments. A high-speed camera is used to capture the images of the keyhole and molten pool, which are closely associated with the welding quality. With image processing and feature extraction, seven features are obtained. Then the PCA algorithm is applied to reduce the redundancy of data to accelerate the training process. Finally, the prediction accuracy of three machine learning algorithms under different chosen PCs are compared in detail. Following conclusions can be drawn:

(1) An advanced high-speed visual image capturing system is devised to capture the morphology information of the keyhole and molten pool. Results show that this visible welding tracking system is effective and practical.

(2) The PCA algorithm is used to reduce the data redundancy to enhance modeling efficiency. The performance of different PCs is distinct. Results show that reduce data dimension is helpful to improve prediction accuracy.

(3) The BPNN, RBFNN, and SVR models are established to model the relation between the weld width and the chosen PCs which contains the information of keyhole and molten pool. The prediction accuracy are compared by comparing the MAE, RMSE and PE results. Results indicate that the BPNN model has best performance, and SVR model is better than RBFNN model.

Overall, this work shows that machine learning algorithms can accurately predict welding quality in real time. Among them, BPNN works best, and effective machine learning algorithm is the key to realize intelligent welding. In recent years, deep learning is increasingly exploited in welding process monitoring and process control. Future research should focus on the combination of multi-sensor and deep learning, through powerful big data processing capabilities of deep learning to process the information collected by multiple sensor fusion technology, the efficiency and reliability of welding process monitoring will increase significantly.

Acknowledgements

This research has been supported by National Natural Science Foundation of China (NSFC) under Grant No. 51775203, No. 51721092 and No. 51861165202, the Fundamental Research Funds for the Central Universities, HUST: Grant No. 2016YXMS272.

Conflict of interest

All authors declare no conflicts of interest in this paper.