NGBoost algorithm-based prediction of mechanical properties of a hot-rolled strip and its interpretability research with ANOVA values

1.   Introduction

As a key component of the national economy, the steel industry plays a crucial role in aerospace technology, production and manufacturing, and national defense construction. Its vitality has driven the steady growth of China's economy and society. The extensive production and application of steel products cover many national construction fields, including machinery manufacturing, automobile production, construction equipment, and electrical appliance manufacturing. However, in response to the new era and its development challenges, the steel industry is undergoing a historic transformation. With the continuous emergence of new situations, formats, and models, the steel industry has entered a new stage of development ^[1].

As an important metal material, hot-rolled strip steel plays a significant role in the engineering field. Predicting its mechanical properties can not only shorten the product development cycle and improve production quality, stability, and efficiency but also help reduce emissions and save resources ^{[2,3,4,5,6,7]}. Although system identification can estimate model parameters well in complex environments, it cannot provide uncertainty quantification ^[8,9]. Accurately predicting the mechanical properties of hot-rolled strip steel is of practical significance for material design, processing, and final product performance. With the continuous advancement of industrial technology and evolving needs, research on predicting the mechanical properties of hot-rolled strip steel has become increasingly urgent. Although traditional experimental methods can provide some performance data, they are limited by factors such as time, cost, and feasibility. Consequently, developing prediction technologies based on theoretical models and computational methods has become a prominent research focus. Reference ^[10] uses a deep belief network combined with a quantile regression loss function and an $\varepsilon$-insensitive loss function to build two models. These models employ unsupervised pre-training of the underlying restricted Boltzmann machine and supervised fine-tuning with the backpropagation (BP) algorithm to enhance prediction capabilities and avoid local optima. Reference ^[11] proposed an improved stacked autoencoder model, combining denoising autoencoders and sparse autoencoders to effectively process noise in the data and address the overfitting problem. Reference ^[12] introduced a hot-rolled strip mechanical prediction model based on extreme gradient boosting (XGBoost), utilizing a genetic algorithm to globally optimize the parameters of the BP neural network, yielding good prediction results. Reference ^[13] described a method based on a support vector machine quantile regression model. By exploring the relationship between the chemical composition and process parameters of the strip and the tensile strength, the least squares fitting algorithm was used to solve the parameters, achieving a smaller prediction error. Reference ^[14] introduced a method that combines the random forest algorithm with mechanistic modeling to establish a performance prediction model by ranking the importance of each factor and gradually adding independent variables, thereby improving the accuracy of the prediction model.

Traditional methods for predicting mechanical properties mainly rely on physical models and empirical formulas. Although these methods can reflect the characteristics of materials to some extent, they often suffer from issues such as complex models, high computational demands, and limited prediction accuracy. With the development of big data technology and machine learning algorithms, data-driven prediction methods have gradually become a research hotspot. By analyzing and modeling large datasets from the production process, these methods can predict the mechanical properties of materials more accurately, thereby guiding production practices more effectively.

NGBoost (natural gradient boosting) is an emerging probabilistic prediction model based on the gradient boosting framework that can perform both regression and classification tasks simultaneously. Unlike traditional gradient boosting methods, NGBoost not only provides prediction values but also measures the uncertainty of these predictions, offering high flexibility and robustness ^[15]. Reference ^[16] studied the application of NGBoost in the field of civil engineering and verified the feasibility of the algorithm in strength prediction and damage classification. Reference ^[17] developed a bond strength prediction model for reinforced concrete structures based on NGBoost and provided an assessment of safety risk uncertainty. Reference ^[18] applied NGBoost for uncertainty estimation of long-term creep fracture life prediction in specific steel. Reference ^[19] employed NGBoost for seismic vulnerability analysis of building structures, thereby reducing model uncertainty and enhancing model prediction accuracy. In the field of materials science, using the NGBoost model to predict the mechanical properties of hot-rolled strip steel can effectively combine multi-dimensional feature information, such as chemical composition and process parameters, to deliver more accurate and reliable prediction results.

The main purpose of this study is to use the NGBoost algorithm to predict the mechanical properties of hot-rolled strip steel and verify its effectiveness and stability through experiments. This paper first introduces the data collection and preprocessing methods, followed by a detailed description of the feature engineering process and model construction method. The model is then trained and evaluated, with the prediction results assessed using the average coverage error (ACE) and prediction interval absolute width (PIAW). The research results demonstrate that the prediction model based on NGBoost exhibits excellent performance in terms of accuracy and stability, providing an effective solution for predicting the mechanical properties of hot-rolled strip steel.

2.   Prediction of steel mechanical properties based on the NGBoost algorithm

2.1.   Prediction of mechanical properties of hot-rolled strip steel

As shown in Figure 1, the production of hot-rolled strip steel involves several key processes, including raw material preparation, heating, and rough rolling. Starting from raw material preparation, the steel undergoes rough rolling after being heated to a suitable temperature, forming a thick strip of steel plate. This is followed by finishing, which thins the plate and improves surface quality and dimensional accuracy. The cooling process needs to control the speed to regulate the structure and performance of the steel. After cooling, the steel is cut, possibly subjected to rust removal, and finally coiled for easy transportation and storage.

The mechanical properties of steel materials are usually measured by indicators such as yield strength, tensile strength, and elongation. Yield strength refers to the stress level at which steel materials begin to undergo plastic deformation, marking the transition from the linear elastic stage to the plastic deformation stage during a tensile test. Tensile strength refers to the maximum tensile stress that the material can withstand under tension. Elongation reflects the ductility of the material; the greater the elongation, the better the ductility of the steel. These three performance indicators are among the most critical parameters in steel materials and must be strictly controlled in accordance with relevant standards during the production process ^[12].

Prediction of the mechanical properties of hot-rolled strip steel involves forecasting and evaluating the material's performance during its use. By assessing these properties under various working conditions, engineers and designers can better understand the stress conditions that materials will experience, allowing them to optimize design schemes and enhance structural safety and reliability. Predicting the mechanical properties typically considers factors such as the material's characteristics, working conditions, and stress distribution. This can be achieved through numerical simulations, experimental testing, and empirical formulas.

2.2.   Construction of the steel mechanical properties prediction model based on the NGBoost algorithm

The NGBoost algorithm is a new ensemble algorithm proposed by Andrew Ng's team in October 2019, also known as the natural gradient-based boosting method. This ensemble learning method has a unique feature, namely, it can directly generate a complete probability distribution in the output space, so that probability can be used to predict uncertainty ^[20].

In NGBoost, base learners usually use decision trees or other simple regression models. Each base learner learns the residuals (i.e., the difference between the current model prediction and the actual value) during training and fits these residuals to gradually improve the prediction accuracy of the overall model. Assuming that the mechanical properties of hot-rolled strip steel follow a normal distribution, choosing a suitable probability distribution is crucial to accurately reflect the characteristics of the data and the performance of the model. The model fits the data by optimizing the parameters of these probability distributions (such as mean and standard deviation).

Assume that the prediction $y|x$ for a new input $x$ is in the form of a conditional probability distribution ${{P}_{\theta }}$, where the parameter $\theta$ is obtained by combining the outputs of $M$ base learners with the initial value, and $\theta$ completely determines the probability prediction $y|x$,

A proper scoring rule $\mathcal{S}$ will take as input the predicted probability distribution $P$ and the actual observed outcomes $y$ and assign a score to each prediction such that the true distribution of outcomes has the best score expectation if and only if it satisfies:

where $Q$ represents the result of the true distribution and $P$ represents any other distribution. During the training process, an appropriate scoring rule is used as the loss function to encourage the model to output calibrated probability predictions. In addition, it is necessary to restrict the parameter family of probability distributions and use its parameters to determine a specific distribution. In this case, the most commonly used scoring rule is the logarithmic score $\mathcal{L}$, also known as maximum likelihood estimation (MLE):

First, a dataset containing mechanical property data of hot-rolled strip samples is collected. This dataset includes tensile strength, yield strength, and elongation as target variables, along with various chemical compositions and process parameters as feature variables related to these mechanical properties.

Next, the NGBoost algorithm is employed to build a prediction model. NGBoost enhances prediction performance by iteratively fitting weak prediction models. In each iteration, it adjusts the model parameters to minimize losses based on the gradient direction of the negative log-likelihood loss function. The NGBoost model is used to predict the target variable by learning the relationship between the input features and the target variables.

Finally, the models prediction results are analyzed, and the prediction process is explained as needed. The feature importance information provided by NGBoost is used to identify which features most significantly contribute to the prediction of mechanical properties. This helps in gaining a deeper understanding of the relationship between the features and the mechanical properties of hot-rolled strip steel.

The input of the NGBoost algorithm for hot-rolled strip mechanical property prediction based on MLE and the natural gradient is the actual test data $D = \left\{ {{x}_{i}} \right., \left. {{y}_{i}} \right\}_{i-1}^{n}$ of a steel plant. Among them, $n$ represents the total data sample size, the subscript $i$ represents the $i$-th sample (the same below), the number of iterations is $M$, the learning rate is $\eta$, and the loss function is $\mathcal{L}$. Assume that the mechanical properties of hot-rolled strip steel obey the normal distribution with parameter $\theta = \left(\mu, \sigma \right)$, where $\mu$ represents the mathematical expectation and $\sigma$ represents the standard deviation.

2.3.   Calculation of prediction interval of the steel mechanical properties

According to the prediction probability distribution of the mechanical properties of hot-rolled strip steel, the prediction interval of the mechanical properties with a confidence level of ${{I}_{\alpha }} = [{{L}_{\alpha }}, {{U}_{\alpha }}]$ can be obtained, where ^[21]:

2.4.   Implementation process

The entire implementation process is shown in Algorithm 1, including data collection and preparation, feature engineering, etc.

3.   Interpretation of the mechanical property prediction results based on the ANOVA value

3.1.   Definition of the ANOVA value

Analysis of Variance (ANOVA), also known as the F test, is a statistical method presented by Sir Ronald Fisher. It is used to compare whether there are significant differences between the means of two or more samples ^[22]. As a commonly used statistical method, ANOVA analyzes the impact of different factors on data variation and determines which factors have a significant effect ^[23].

In predicting the mechanical properties of hot-rolled strips, multiple influencing factors, such as chemical composition and rolling process parameters, can affect the mechanical properties of the strips. Therefore, ANOVA can be used to determine which factors have a significant impact on the mechanical properties and to explore interactions between factors. For example, factors like chemical composition and rolling temperature can be used as influencing factors, while mechanical properties (such as yield strength) can be used as response variables for ANOVA.

The ANOVA value is a result of multi-factor analysis of variance and measures the influence of each factor on the variable. A larger ANOVA value indicates a more significant impact of the factor on the variable; conversely, a smaller value suggests a less significant impact. In predicting the mechanical properties of hot-rolled strip steel, the ANOVA value can identify which factors have a crucial influence on mechanical properties and their relative importance.

Studying the ANOVA value provides an in-depth understanding of the relationship between the mechanical properties of hot-rolled strip steel and various factors. This analysis helps optimize process parameters and improve the prediction accuracy and stability of the mechanical properties.

3.2.   Calculation of the ANOVA value

The main factors affecting the mechanical properties of hot-rolled strip steel are expressed by ANOVA ^[24,25], which represents the main factors affecting the mechanical properties of hot-rolled strip steel. The specific analysis is as follows:

(1) Calculate the mean of each sample

Calculate the average value of each sample, the sample average value corresponding to the influencing factor and the total mean of all observations. The calculation formula is as follows:

where ${{n}_{i}}$ is the sample size of the $i$-th influencing factor, ${{x}_{ijk}}$ is the $k$-th observed value of the $j$-th type of influencing factor under the $i$-th performance indicator, ${{\bar{x}}_{ik}}$ is the sample mean of the $k$-th type of influencing factor under the $i$-th performance indicator, and ${{\bar{x}}_{k}}$ is the total mean of all observations under the $k$-th performance indicator.

(2) Calculate the sum of squares of each error

${{E}_{SSAk}}$ is the sum of squares of the errors between the mean of each group ${{\bar{x}}_{ik}}$ and the total mean under the $k$-th performance index, ${{E}_{SSEk}}$ is the sum of squares of the errors between the sample data of each group and its group mean under the $k$-th performance index, and the calculation formula is as follows:

(3) Calculate statistics

In order to eliminate the effect of the number of observations on the size of the error sum of squares, ${{E}_{SSAk}}$ and ${{E}_{SSEk}}$ need to be averaged. The ${{E}_{SSAk}}$ degrees of freedom is $i-1$, and the degrees of freedom is $n-i$. The mean square and statistic calculation formula are as follows:

where $F$ is the test statistic. The larger of the value of $F$, the more significant the influence of this influencing factor on the mechanical properties of the strip steel.

4.   Case analysis

This paper takes the hot-rolling production line data of a steel plant as an example to build a prediction model to predict the mechanical properties of hot-rolled strip steel.

4.1.   Experimental data and prediction algorithm settings

The hot-rolling data used in this article includes 138 features and 6300 samples, and a description of all features is visible in Table 1. Due to the harsh environment of the data source, data preprocessing is performed first. Data preprocessing plays a vital role in the entire data analysis process because it is directly related to the accuracy of subsequent model building and performance prediction results. In order to ensure the robustness of the model, missing values processing, outlier processing, and feature selection become three important links.

First, missing values in the data are a common problem, especially in industrial production environments where some sensors may fail to record data due to malfunction or other reasons. Missing values that are not properly handled may lead to bias in model training or even cause the algorithm to fail to operate properly. We adopt the method of missing value deletion for data rows with few records and a large number of missing values to ensure data quality. After processing, 5834 samples were obtained with the same dimensions.

Second, outliers are data points that are significantly different from the majority of the data and are usually caused by measurement errors or extreme operating conditions. In order to identify and deal with outliers, we use the 3-sigma principle, which assumes that the data follow a normal distribution, and data points that are more than 3 standard deviations away from the mean are considered outliers and are removed. This prevents outliers from adversely affecting the model. In this way, we obtained 5331 samples.

Finally, production data often contain a large number of features, some of which have a weak correlation with the prediction target and may even introduce noise. Therefore, feature selection is a very critical step in data preprocessing. We utilize recursive feature elimination to screen features. By recursively training the XGBoost model, evaluating the importance of features, and gradually eliminating unimportant features, the most representative features are finally selected for model training. The influencing factors selected for constructing the prediction model are shown in Table 2.

After preprocessing, a sample set consisting of 5331 data with a dimension of 16 is obtained, which is divided into a training set and a testing set in a 7:3 ratio. For the training set, a random search strategy based on the Scikit-learn library is used as the optimization method for NGBoost hyperparameters using decision trees (DT) as regression models. Through 10-fold cross-validation, optimal parameter selection is obtained as shown in Table 3.

4.2.   Steel performance prediction and evaluation index

ACE and PIAW are used to evaluate the prediction performance of the algorithm. ACE reflects the reliability of the prediction, while PIAW reflects the sensitivity of the prediction ^[26].

ACE is defined as the difference between the empirical coverage probability (ECP) and the nominal coverage probability (NCP) where the actual value falls within the predicted range. The calculation formula is shown in (4.1).

where ${{D}_{ACE}}$, ${{D}_{ECP}}$, and ${{D}_{NCP}}$ represent the values of ACE, ECP, and NCP, respectively. ${{U}_{\alpha }}$ and ${{N}_{\alpha }}$ correspond to the upper and lower limits of the prediction interval. $N$ represents the number of data points. $y$ represents the actual value of the mechanical properties of each data point, and ${{\varepsilon }_{\alpha }}$ represents the indicator function, which is 1 only when the actual value of the mechanical properties is within the prediction interval, and otherwise it is 0. The smaller the absolute value of ACE, the higher the credibility of the prediction interval.

PIAW is defined as the average width of the prediction interval, and its calculation is shown in (4.2).

The smaller the value of PIAW is, the better the prediction result is at aggregating uncertain information.

4.3.   Analysis of the prediction results of the steel mechanical properties

As can be seen from Figures 2–7, the NGBoost algorithm has the largest error in predicting elongation and the smallest error in predicting tensile strength, and converges very quickly. After about 175 iterations, the loss function changes slightly. ACE and PIAW are used as probability prediction indicators, and the root mean squared error (RMSE) and R-squared (R²) are used as single-point prediction indicators (the mean is treated as a predict value). Table 4 shows the performance of the algorithm on the training set and test set.

4.4.   Prediction and comparative analysis of the mechanical properties of steel

Since NGBoost not only provides probabilistic predictions but also gives point estimates, therefore, we perform a comparison between the probabilistic prediction metrics ACE and PIAW, and likewise for the RMSE and R² metrics for mean prediction.

Remark 1. Since most regression models do not provide probabilistic predictions, here we use residual analysis to construct prediction confidence intervals. Specifically, the regression model is trained with a training set, the residuals of the predicted and true values are computed, the distribution of the residuals is fitted to obtain the mean and variance, and the confidence intervals are constructed for the predicted values of the new test data.

The regression performance and probability prediction performance of the NGBoost algorithm are compared with that of the traditional gradient boosting (GB) algorithm, the Lasso regression algorithm, and the DT regression algorithm. The results of the RMSE and R² metrics for these algorithms are shown in Table 5. According to the probability distribution prediction results of yield strength, tensile strength, and elongation using different algorithms, the ACE and PIAW of the prediction interval are calculated at the confidence levels of 80%, 90%, and 95%, respectively, and the results are shown in Tables 6–8.

From the results in Tables 6 and 8, it can be seen that when the confidence level is set to 80%, 90%, and 95%, the NGBoost algorithm obtains the smallest absolute ACE and PIAW values compared with other algorithms. Therefore, the reliability of the NGBoost algorithm is higher than that of other algorithms.

According to the results in Table 7, when the confidence level is 80%, the reliability of the DT algorithm is relatively higher than that of other algorithms (the absolute value of ACE is the smallest), but compared with the NGBoost algorithm, its prediction interval is wider (the PIAW value is larger), indicating that its ability to aggregate uncertain information is relatively weak. Specifically, the NGBoost algorithm not only performs well in terms of reliability, but also has a narrow prediction interval (a smaller PIAW value), indicating that it has a strong ability to aggregate uncertain information.

In order to verify the performance of the NGBoost algorithm in the probability prediction of mechanical properties, this paper uses this algorithm and other methods to predict the probability distribution of mechanical properties, and calculates the prediction intervals at different confidence levels (50%, 80%, 90%, 95%, 98%, 99%). In order to see the comparison results of different algorithms more clearly and intuitively, the tabular data is presented in the form of a histogram, and the results are shown in Figures 8–13. It can be seen from Figures 8–10 that at different confidence levels, the absolute value of ACE of the NGBoost algorithm is small, showing higher reliability, and according to Figures 11–13, it can be seen that the PIAW of the NGBoost algorithm is small, showing better sensitivity. In summary, the NGBoost algorithm not only provides higher reliability, but also shows better sensitivity.

Remark 2. To prevent model overfitting, we adopt various measures, including cross-validation and an early stopping mechanism, that stops model training when the performance of the validation set is no longer improving. These methods effectively improve the generalization ability of the model.

4.5.   Importance analysis of the characteristics affecting the mechanical properties of steel

After performing the ANOVA, the results can be interpreted in terms of F-values and p-values, where the F-value represents the degree of difference in the variance between different groups. The larger the F-value, the more significant the difference in mean values between different groups, and the p-value indicates the probability of whether the observed results are caused by random factors. Generally, if the p-value is less than the significance level (such as 0.05), the null hypothesis is rejected and the difference is considered significant.

The F-values and p-values of different mechanical properties are shown in the Figures 14–16 below:

It can be seen from Figure 14 that the top six characteristics with the greatest influence on yield strength are Ti, Nb, Mn, Si, C content, and the average finishing outlet temperature. It can be seen from Figure 15 that the top six characteristics with the greatest influence on tensile strength are Ti, Nb, Si, Mn, C, Mo content. It can be seen from Figure 16 that the top six characteristics with the greatest influence on yield strength are Si, Mn, Ti, C, Nb content, and the average finished product thickness.

The results of ANOVA show that chemical composition is a key factor affecting the mechanical properties of steel, especially for improving the tensile strength and toughness of steel. These results have the following references for engineering decisions in the steel industry.

1) Chemical composition optimization: Engineers can adjust the ratio of alloying elements based on the analysis results to balance the mechanical property requirements under different working conditions. For example, increasing the carbon content can increase tensile strength, but may reduce ductility. Therefore, by optimizing the chemical composition, engineers can optimize the material properties according to the usage scenarios of the final product.

2) Production cost control: Some rare alloying elements (e.g., chromium, nickel) are costly, and ANOVA can help producers decide whether to use these elements based on their actual needs, thus achieving a balance between performance and cost.

3) Quality control and consistency: By analyzing chemical compositions, manufacturers can better control the consistency of materials during the production process and ensure that the final product meets requirements across batches and working conditions.

5.   Conclusions

To predict the mechanical properties of hot-rolled strip steel, analyze the composition (chemical elements) of the strip, and assess the influence of process parameters during rolling, this paper proposes a prediction method based on NGBoost and ANOVA values. The actual production data from a steel plant was used for verification. The results show that:

1) The proposed method can effectively predict the mechanical properties of hot-rolled strip steel, demonstrating fast convergence speed and good prediction performance.

2) The influencing factors of the mechanical properties were analyzed using the ANOVA value, revealing that the main factor affecting the mechanical properties is the chemical composition.

Author contributions

Hongyi Wu: Conceptualization; Jinwen Jin: Methodology; Zhiwei Li: Validation. All authors have read and agreed to the published version of the manuscript.

Conflict of interest

The authors declare that there is no conflict of interest.