Probabilistic invertible neural network for inverse design space exploration and reasoning

1.   Introduction

A variety of predictive algorithms have been developed to explain the forward mapping of input parameters x to measurable output parameters y. For some engineering applications, the inverse mapping from y to x is of paramount importance. Especially in the field of product design, inverse prediction based on specified output performance is required to obtain feasible design parameters. The design of industrial equipment represents a tricky and challenging task that encompasses multi-disciplinary knowledge. To optimize multiple performance objectives simultaneously, engineers need to perform hundreds of design iterations.

As a widely used power device in industry, the gas turbine incorporates principles from the disciplines of mechanics, materials, thermodynamics and aerodynamics which leads to a bulky and slow design process. This demands designers to comprehensively consider the performance of modules (e.g., combustors) as well as components (e.g., blades) to ensure overall stability and efficiency. Due to the fact that inverse design outputs the distribution of design parameters from the desired response, the interpretability and efficiency of the design space exploration could be improved comparing with that from only the forward design process. A range of inverse design methods have been developed which fall into two categories roughly: optimization-based methods and inverse process-based methods ^[1].

Optimization-based methods refer to the process searching for the solution that makes the target performance optimal in the design parameter space under the constraints. It is essentially a forward process, where the system response/performance obtained by simulations or experiments is used to adjust the design parameters reversely. Classical meta-heuristic algorithms including genetic algorithm and particle swarm algorithm are widely used in model update design of steam turbines and other equipment ^[2,3]. In special design cases, some deterministic algorithms ^[4,5] have been proven to achieve optimal solutions equivalent with heuristics in a much shorter time. These methods involve multiple iterations of the forward simulation process which might be computationally expensive. Although the simulation process can be replaced by a surrogate model to reduce computation time ^[6], these methods might fall into local optimum dilemmas readily. The most critical point is that there is a many-to-one mapping relationship between design parameters and performance parameters. Methods based on optimization cannot capture these possible design candidates, which is urgently required by engineers. Bayesian inference methods present the posterior distribution of parameters, thus achieving the goal of describing non-unique design parameters.

Bayesian inference methods mainly include the Markov chain Monte Carlo (McMC) sampling approach ^[7,8] and variational inference (VI) ^[9]. McMC obtains multiple sets of possible design parameters by sampling the posterior distribution function, while the mean and variance of these data are derived to describe the parameters' Probability Density Function (PDF). McMC is widely used ^[10,11,12] to approximate the empirical PDF for the parameters of interest. VI searches for simple distributions that approximate the target distribution (usually multi-modal) in a predefined family of distributions by optimizing the KL divergence. Attributed to the fact that VI is using optimization rather than random sampling, it improves the efficiency of solving the parameter distribution to some extent. In ^[13,14], VI significantly helps the design of circuits or communication systems. Still, their computational efficiency is not high enough.

Neural network-based methods facilitate real-time prediction of the inverse design. Neural networks (NNs) are flexible nonlinear mappings trained to efficiently predict outcomes regardless of the actual physical processes. Taking advantage of the rapidity of NNs, they are extensively used in inverse scattering problems ^[15]. The most typical ones are: ^[16] using fully connected neural networks (FCNN) to recover the shape parameters of unknown scatterers; ^[17] generating the obstacle appearances in reverse on the basis of two-layer long short-term memory (LSTM) neural networks; ^[18] proposing a parameter inversion model by a neural network (PIMNN), which captures the trajectory of moving point source to complete the subsequent predictions. Moreover, NNs also serve as surrogate models to achieve inverse design optimization ^[19,20,21]. However, these methods fail to provide an uncertain estimate, which means that NNs only seek one result for each input. In ^[22,23,24], multiple groups of NNs are trained simultaneously using Bayesian neural networks or dropout to estimate the uncertainty of input parameters, but they cannot acquire complex parametric distributions. Recently, various conditional generative models based on generative adversarial networks (GAN) have been widely used in inverse design ^{[25,26,27,28]}. These emerging models have achieved many promising effects, while the training process suffers from instable convergence.

To fully obtain the PDF of input parameters, the invertible neural network (INN) model is proposed ^[29]. INN is a bidirectional mapping network based on affine coupling blocks ^[30,31], considered as an excellent approach for solving inverse problems. The forward process of an INN predicts the output from the input while the inverse process derives the distribution of the input parameters. The key information of its inverse process is captured by the latent variables. INN shines not only in the field of computer vision ^[32,33], but also has distinctive achievements in the field of inverse design ^{[1,34,35,36,37]}.

The forward process of the standard INN only gives the prediction result without presenting uncertainty. Moreover, its inverse process is usually acquired in a qualitative manner and not accurate enough for design purposes. To address this issue, we propose the P-INN with integrated epistemic uncertainty and aleatoric uncertainty. In this way, the outcome of the forward process carries uncertainty, while the posterior distribution of the inverse process incorporates epistemic uncertainty. We also design a bidirectional loss during training to improve the inverse prediction accuracy.

In the following sections, Section 2 describes the proposed method, including the architecture of P-INN and the realization of bidirectional prediction. Section 3 evaluates P-INN with two mathematical functions with varying dimensionality regarding prediction accuracy and reliability of prediction uncertainty. The case of inverse design of turbine blades in engineering is presented in Section 4. Section 5 concludes the work with major findings.

2.   The proposed probabilistic Invertible neural network

The proposed P-INN is shown in Figure 1. The main part of the framework is the INN, which consists of a series of affine coupling blocks. Model uncertainty includes epistemic uncertainty (EU) and aleatoric uncertainty (AU). Here, EU is obtained by the Monte Carlo dropout method ^[38] while AU is acquired by the additional noise channel, denoted as $\boldsymbol{\sigma }$. In order to make the framework more general, it is necessary to add data preprocessing and postprocessing.

2.1.   Architecture of INN

The INN architecture is shown in Figure 2, whose basic unit consists of an invertible block with two complementary affine coupling layers. In this work, we adopt a Real NVP architecture ^[31], which is superior for learning the distribution of data with stable convergence.

Denote the input vector of each invertible block as u, and divide u into two parts as u₁ and u₂. ${s}_{i}$ and ${t}_{i}$ are trainable affine functions, where $i\in \left\{\mathrm{1, 2}\right\}$. The input vector group (u₁, u₂) is converted by them to obtain the corresponding output vector group (v₁, v₂). In this work, ${s}_{i}$ and ${t}_{i}$ are assigned as fully connected neural networks with ReLU for the activation function, while these shallow neural networks will be considered as subnetworks of INN. v₁ and v₂ form the output vector v. The forward process of an invertible block is described as follows:

where $\mathrm{☉}$ represents the element product of the matrix. Likewise, the reverse process expressions are listed below:

It is worth noting that the "+" and "☉" operations in Eq (1) already provide the reversible effect of INN, thus the ${s}_{i}$ and ${t}_{i}$ functions allow arbitrary mappings without requiring them to be invertible. An adequate volume of invertible blocks is connected in series to deepen the network; a certain level of zero vectors is replenished at the input and output to enhance the predictive capability of the network. Because each invertible block requires the same dimensionality of inputs and outputs, the input and output dimensions of INN also need to be consistent as shown in Figure 1 and Eq (3):

where x is the input parameter, y is the output parameter, σ denotes the AU corresponding to y, and z denotes the latent variable. To enhance the interaction between the dimensions of parameters, each invertible block is followed by a permutation block in this work, which is used to randomly shuffle the dimensions of the parameters.

2.2.   Probabilistic invertible neural network (P-INN)

The standard INN can generate an exhaustive posterior distribution of input parameters based on specific output parameters, which is extensively used for solving inverse problems ^[29]. However, INN has several problems: it may not converge for some regression tasks; results obtained by the forward process lack uncertainty; the distribution achieved by the inverse process is not accurate sufficiently. To address these issues, we propose the probabilistic invertible neural network (P-INN) framework shown in Figure 1, which provides uncertainty assessment while improving the bi-directional prediction accuracy.

We first review the knowledge about uncertainty ^[39]. Uncertainty is broadly divided into epistemic uncertainty (EU) and aleatoric uncertainty (AU) ^[40]. EU describes the uncertainty of model parameters, which is due to the fact that the prediction can be affected by factors such as poor training and insufficient training data. AU refers to the error caused by the intrinsic noise of data. ^[24] proposed that the AU may be predicted directly using the likelihood loss, which will be described in detail in Section 2.3. Common methods for EU acquisition include model fusion as well as Bayesian neural networks (BNN). The model fusion-based approach aims to train multiple network models on the same dataset; the predictions of these models are finally synthesized. The weight of each parameter in BNN is a distribution, so weights are sampled to get different results when predicting. The Monte Carlo dropout approach is a simple implementation, which we adopt in this work. The forward prediction distribution is drawn according to Eq (4):

where D is the training dataset, $p\left(y, z|x, \theta \right)$ is the likelihood function, and $q\left(\theta , D\right)$ is the posterior function of the weight parameters.

T dropout samples are produced to approximate the distribution of weights, and use Monte-Carlo to estimate the first moment of the prediction function:

where ${W}_{i}^{t}$ represents the parameter matrix of the i-th layer, and it is assumed the INN has a total of L layers. The second moment is solved in a similar way. In practice, we only need to activate dropout in forward process. EU is represented the calculated variance of T predictions.

The inverse process takes response y, predefined σ, and the sampled z as input, and derives the x. Set $f = {g}^{-1}$, then ${f}^{-1}\left(y, z, {\sigma }_{c};\theta \right)$ is denoted as $g\left(y, z;\theta \right)$. The latent variable z is set to be the standard normal distribution $z~p\left(z\right) = N(0, \boldsymbol{I})$. Depending on the specific y each set of z is sampled from the distribution $p\left(z\right)$ in order to derive the corresponding x. A one-to-many mapping of the inverse process (a set of y generates multiple sets of x) is implemented by sampling z multiple times. Define the complete distribution of results for inverse process as shown in Eq (6):

where the denominator part is the Jacobian determinant embodied by the inverse process. The dropout of the fully connected layer is activated where the generated x is combined with EU.

2.3.   Loss function

The loss function is formulated with 6 terms from 2 categories. The loss terms in the first category aim for improving the forward prediction accuracy, including the mean square error (MSE) of output responses, the likelihood of output responses and the distribution term of latent variables. The loss terms in the second category are beneficial to improve the inverse prediction accuracy, including the distribution term of input parameters, reconstruction term and bidirectional term. Each loss term has a specific contribution as detailed in the following paragraphs.

MSE term of output response. The purpose of this loss term is to make the predicted output response and true value as identical as possible, which is the main goal of forward process. The MSE loss term is defined in Eq (7):

where ${ŷ}^{\left(i\right)}$ is the forward predicted output response value, ${{y}^{*}}^{\left(i\right)}$ is the ground-truth response value, θ refers to the weight parameters that INN needs to learn, and N denotes the number of training samples.

Likelihood term of output response. This loss term is proposed to obtain the AU of the response. AU measures the noise within the data, which we characterize directly using an additional σ channel. In general, the input dimension is much larger than the output dimension. INN requires that the input dimension be consistent with the output dimension, which will be supplemented with the zero vector. The σ channel replaces part of the zero vectors. Thus, the loss function of this term is defined as follows:

In Eq (8), the dimension of ${\sigma }^{\left(i\right)}$ is aligned with the dimension of the output response.

Distribution term of latent variable. The latent variable z can be arbitrarily distributed, and we take $z~N(0, 1)$ to simplify the calculation. In the inverse process, the probability distribution of the input parameters is acquired with sampled z. The distribution term aims for measuring the difference between the distribution of forward predictions and the distribution of the training samples. The loss function is defined as follows:

where $\stackrel{~}{z}$ and ${z}^{*}$, respectively, represent the predicted and genuine value of the latent variable; $q\left(ŷ, \stackrel{~}{z};\theta \right)$ is the distribution for the result in forward process; $p\left({y}^{*}\right)$ is the prior knowledge of output response from training samples; $p\left({z}^{*}\right)$ is the standard normal distribution. MMD refers to Maximum Mean Discrepancy, which is an effective way to compare differences between distributions.

Distribution term of input parameters. This term aims to narrow the difference between the inverse prediction distribution and the prior knowledge of input parameters from training samples, which can improve the accuracy of inverse prediction to a certain extent. The loss term is defined in Eq (10):

where $\stackrel{~}{x}$ denotes the prediction result in inverse process; ${x}^{*}$ refers to the authentic input data; $q\left(\stackrel{~}{x};\theta \right)$ is the distribution of the prediction result; and $p\left({x}^{*}\right)$ is the prior knowledge of inputs from training samples.

Reconstruction term. In the inverse process, a set of input parameters is generated for each set of z samples. Since the value of z is infinite, the result of INN backward prediction has a high degree of freedom for a given set of the output response. Design and reasoning of critical engineering systems require accurate posterior distribution of input parameters from a given system response. This loss term ensures that the same input parameters are predicted as identical as possible when sampling similar z. The loss term is defined in Eq (11):

where ${f}^{-1}\left(y, z, {\sigma }_{c};\theta \right)$ refers to the inverse process, and ${\sigma }_{c}$ takes a small constant because the given y is a ground truth value. Noise term is added to the predicted latent variable form increased robustness.

Bidirectional term. The bidirectional loss is proposed for further improvement of the inverse prediction accuracy. The distribution of input parameters is captured by the sampled latent variable z, but there is no guarantee that the distribution can precisely reflect the true input parameters. It is difficult to directly measure the gap between predicted values and the ground-truth values. We propose an indirect measurement for the distribution of inverse prediction. The predicted input parameters are first converted using the trained forward process h(x) where we then calculate the MSE between the transformed and true values. Here $\mathrm{h}\left(\mathrm{x}\right)$ is set to be $f(x;\theta )$ and the loss term is defined as in Eq (12):

where $f(x;\theta )$ represents the forward process. Different from reconstruction loss term, ${z}^{\left(i\right)}$ is sampled from the standard normal distribution. The bidirectional loss forces all potential predicted input parameters to be reducible to the specified value.

Total loss. The total loss is the sum of all the above losses, as in Eq (13):

where ${\lambda }^{i}$ represents the weight of each loss. After data preprocessing, y is normalized to be between 0 and 1. It is obvious that the MSE term, Likelihood term, and Bidirectional term all correlate directly or indirectly with y. Consequently, the order of magnitudes for these terms is the same. Meanwhile, the experiments indicate that the results of the two distribution terms are also appropriate. We consider that these five terms are balanced for model training, taking their weights as the one here. To adjust the Reconstruction term, its coefficient is the reciprocal of the square for the value range taken by the input data, which has a normalization-like effect.

3.   Numerical evaluations

In this section, two mathematical problems are adopted to evaluate the proposed method. Experiments show that the method has better results in multi-objective predictions and high-dimensional input situations.

3.1.   Multi-objective problem

Mathematical settings. To test the performance of P-INN on the inverse problem, we design a multi-objective regression task. Consider an example of two-dimensional input and two-dimensional output. The input x is written as [x₁, x₂], and the output y is written as [y₁, y₂]. The input domains are ${{\Omega }_{x}}_{1}ϵ[-\mathrm{5, 10}]$, ${\Omega }_{{x}_{2}}ϵ\left[\mathrm{0, 15}\right]$ which equipped with a uniform probability density $p\left({x}_{i}\right)$. The two-objective functions ${y}_{1}$ and ${y}_{2}$ are modified from the branin and hyperbolic paraboloid function. Moreover, the curved surfaces are shown in Figure 3a, b respectively, where the outputs have been normalized. The function expressions are shown in Eq (14):

where $a = 1$, $b = \frac{5.1}{4{\pi }^{2}}$, $c = \frac{5}{\pi }$, $r = 6$, $s = 10$, $t = \frac{1}{8\pi }$; $u = 2$, $v = 7$, $m = 2.25$, $n = 400$. Branin function is a commonly used complex surface for optimization problems, while hyperbolic paraboloid function is a universal surface in structural design, so we select them as the test functions. We train the P-INN with 6 invertble blocks and the ${s}_{i}$ and ${t}_{i}$ functions of the invertble block both use a 5-layer NN. All hidden layers are followed by a dropout layer with a dropout coefficient of 0.1. Train on datasets of size 50,100 and 150 while optimizing for 3000 iterations using the Adam optimizer.

Benchmarks. To evaluate the superiority of the proposed method, we compare it with the standard INN. In fairness, the standard INN has the same number of reversible blocks, and the ${s}_{i}$ and ${t}_{i}$ of reversible blocks are also consistent with P-INN. We also train a forward 5-layer DNN to compare the prediction accuracy of the proposed method.

Evaluation metrics. To intuitively evaluate the performance of P-INN, we use two types of metrics to measure the quality. The first type measures the prediction accuracy, including the root mean square error (RMSE). However, since the reverse prediction has multiple unknown truth values, the inverse RMSE cannot be calculated. We use an indirect approach: convert the obtained input parameters through the ground-truth functions; then compute the RMSE between transformed values and true output parameters. The second type measures uncertainty interval includes Prediction Interval Coverage Probability (PICP) and PI Normalized Averaged Width (PINAW). PICP evaluates the ability of the uncertainty interval for capturing the true value, as follows:

where ${C}_{j} = 1$ if the true value is in the uncertainty interval, otherwise ${C}_{j} = 0$. PICP ranges from 0% to 100%. For a specified probability of $100(1-\alpha )\%$, the uncertainty can be considered reliable if $PICP\approx 100(1-\alpha )\%$. PINAW evaluates the width of the uncertainty, as a percentage of the target range. The following definitions are given:

where $R$ is the range of the target variable and ${L}_{j}$ is the range of the uncertainty interval. The predicted uncertainty behaves better with a lower PINAW. Similarly, since the true value of the inverse is difficult to determine, we transform the inverse predicted value into the output space through the ground-truth functions, and then perform uncertainty evaluation.

Forward discussion. The prediction accuracy and uncertainty estimates of the forward process are analyzed from Tables 1 and 2, where the data that perform outstanding are bolded. When the training dataset size is small (such as size 50), the prediction accuracy of P-INN is lower, but still more accurate than INN. If trained on a sufficient dataset, the proposed method will even outperform DNNs. It is intuitive that the forward RMSE of P-INN is only 0.0199 and 0.0338 with 150 sets of training data while the forward RMSE of INN is 0.2021 and 0.2861, which is even considered as a fitting failure. The surfaces acquired by the proposed method are shown in Figure 3c, d while the surfaces derived by DNN are shown in Figure 3e, f. The surfaces fitted by INN are poor so we don't show them. Obviously, although the surfaces obtained by the proposed method are not smooth, they fit the real surface better than DNN. We find that in the case of few data, the PICP of the forward process can still be close to 1 (assuming a reliability of 0.95), but the PINAW is a bit large. This means that the uncertainty interval covers almost all true values, but it is wide so the uncertainty is not entirely convincing. With an appropriate increase in training data (such as a size of 100), PINAW drops below 0.5.

Inverse discussion. Tables 3 and 4 demonstrate the superiority of P-INN in reverse process. Because DNN does not have the capability of inverse prediction, only the proposed method is compared with INN. Experiments show that the inverse prediction accuracy of P-INN is often much higher than that of INN. PICP remains above 0.95 when the training dataset size is greater than 50. Therefore, the obtained uncertainty is considered to be reliable. Meanwhile, PINAW stays under 0.5, which means that the uncertainty interval is narrow enough.

3.2.   High-dimensional input problem

Mathematical settings. We design a high-dimensional input example to show that P-INN is effective in general applications. The weight of an airplane's wing is related to a number of factors that designers must take into account when designing a wing. Therefore, we take the classical wing weight function, which has 10-dimensional inputs, as follows:

Where ${S}_{w}$ refers to the wing area, ${S}_{w}ϵ\left[\mathrm{150,200}\right]$; ${W}_{{f}_{w}}$ represents the fuel weight of the aircraft, ${W}_{{f}_{w}}ϵ\left[\mathrm{220,300}\right]$; $A$ denotes the aspect ratio of the wing, $Aϵ\left[\mathrm{6, 10}\right]$; $\Lambda$ denotes quarter-chord sweep, $\mathrm{\Lambda }ϵ\left[\mathrm{6, 10}\right]$; $q$ denotes dynamic pressure at cruise, $qϵ\left[\mathrm{16, 45}\right]$; $\mathrm{\lambda }$ denotes taper ratio, $\mathrm{\lambda }ϵ\left[\mathrm{0.5, 1}\right]$; ${t}_{c}$ denotes aerofoil thickness to chord ratio, ${t}_{c}ϵ\left[\mathrm{0.08, 0.18}\right]$; ${N}_{z}$ denotes ultimate load factor, ${N}_{z}ϵ\left[\mathrm{2.5, 6}\right]$; ${W}_{dg}$ refers to flight design gross weight, ${W}_{dg}ϵ\left[\mathrm{1700, 2500}\right]$; ${W}_{p}$ denotes paint weight, ${W}_{p}ϵ\left[\mathrm{0.025, 0.08}\right]$. Each input variable is uniformly sampled in its domain. The training parameters are similar to Section 3.1. It should be noted that the order of magnitude difference between the input variables is significant, thus the reconstruction loss must be assigned weights related to the range of input variables. The reverse process is followed by data post-processing, which is implemented by limiting the predicted value within the corresponding domain. Figure 4 presents the loss curves for a given training data set of 100. As this is a high-dimension input problem, the output requires complementary zero vectors to guarantee dimensional alignment, which will contribute to greater values of MSE term and likelihood term at the early stage of training. In the meantime, it can be seen from Eq (8) that the value of likelihood term may be less than 0 when the ${\sigma }^{\left(i\right)}$ is small. Due to the above reasons, we add a constant of 10 to the loss results and then logarithmically process them to obtain better visualization. As shown in Eq (12), the bidirectional term relies on the exact forward process, hence we incorporate this loss after the 400th epoch. It is noticed that all six losses converge and tend to stabilize after the 450th epoch.

Benchmarks and metrics. INN and DNN are provided to compare the prediction results in bidirectional, where their network parameters are analogous to those in the multi-objective problem. The evaluation metric of prediction accuracy is RMSE, while the evaluation metric of uncertainty interval is PICP and PINAW.

Discussion. The comparison of prediction accuracy of different methods is shown in Table 5. The proposed method excels on high-dimensional input examples: the forward RMSE is already the lowest with a dataset size of 50, while the forward process of INN behaves terribly. The forward RMSE and reverse RMSE of the proposed method are 0.0839 and 0.1309 with an increase in training set size to 150. The inverse uncertainty of P-INN is calculated, as shown in Table 6. The final PICP is 0.95 and the PINAW is 0.357, both indicating that the uncertainty is certainly convincing.

Visualization. The input dimension in this example is 10, which means that the performance of the model is unable to be visualized. We use principal component analysis (PCA) to reduce dimensionality of the input. The main idea of PCA is to map n-dimensional features to k-dimensions, which are completely new orthogonal features. The new features are reconstructed on the basis of the original n-dimensional features. The calculation shows that the cumulative contribution rate of the first two eigenvalues is 0.9943, so the original input features can be replaced with new two-dimensional features. The eigenvectors corresponding to the eigenvalues indicate the contribution of the original features to the new features, and the results show that ${W}_{dg}$ plays an important role. Figure 5 shows the mapping relationship between ${W}_{dg}$ and output (all other input parameters are fixed; here the average of each parameter is taken). The generated training data is added with noise that satisfies the standard normal distribution to simulate the real situation, and some training points are shown in Figure 5a. The curve derived by P-INN almost coincides with the real curve, while the curve obtained by INN is unacceptable. The curve fitted by DNN is also not quite satisfactory.

4.   Application to the turbine blade design

4.1.   Turbine blade simulation

Jet engine produces thrust by jetting backwards to achieve forward movement. Gas turbine is a vital component of jet engine because of its light weight, compact structure, and high combustion efficiency ^[41]. Gas turbine engines are divided into four categories: turbojet, turbofan, turboprop, and turboshaft. When a jet engine works, the gas drawn in is compressed by a compressor to the smallest possible optimal volume. The compressed gas pressure is almost 30 times the original, while the temperature rises to 1000℃. It is then sent to the combustion chamber for combustion. The turbine is responsible for extracting energy from the high temperature and pressure gases produced by the combustion chamber. Due to the long-term extreme working environment, turbine blade materials are usually chosen from high-strength, high-melting nickel alloys. Turbine blades expand when heated, creating significant stress at their joints resulting in strains of several millimeters, which can cause friction and even damage between the blade tip and the casing. Therefore, designers must consider the stress and strain of the blade in the normal working state when designing the turbine. It is natural for us to apply P-INN to the inverse design of the blade.

The simulated 3D structure of the turbine blade is shown in Figure 6. In reality, the stresses and strains of the blade are affected by a combination of pressure and thermal effects. We first calculate the stress and strain that are only affected by the pressure while ignoring the thermal effect; then analyze the stress and strain that only has the thermal effect while ignoring the pressure; finally, the simulation model with the combined effect of the two factors is obtained. The stress or temperature heat maps of the three processes are shown in Figure 7. The pressure model parameters include Young's modulus $E$, Poisson's ratio $nu$, the thermal expansion coefficient $CTE$ of the nickel-chromium alloy, and the pressures $p1$ and $p2$ on both sides of the blade; the thermal effect model parameters include the thermal conductivity $kapp$ of the nickel-chromium alloy.

Here, $kapp$ is fixed and the remaining 5 variables are taken as input. Each variable ranges from 80% to 120% of the recommended value, sampling 100 sets of input data evenly. The simulated stress is used as output. 20 sets of data were randomly selected as the test set. The prediction accuracy of P-INN is compared with INN and DNN.

4.2.   Design space exploration with P-INN

The prediction accuracy of different methods is shown in Table 7. It can be found that even if the training set is small, the forward accuracy of the proposed method is already ahead of other algorithms; when given more data to train, the forward accuracy and inverse accuracy of P-INN have reached a satisfactory level (respectively 0.0634 and 0.1052). After sufficient training, INN's forward accuracy is poor but inverse accuracy is acceptable. DNN does not perform well in this case.

The inverse uncertainty predicted by the proposed method is shown in Table 8. The PICP has reached 1 with a training set size of 40, which indicates that the uncertainty interval fully covers the true value. PINAW remains around 0.5 with adequate training datasets. Figure 8 visualizes the uncertainty intervals of the inverse forecast, where the results for P-INN and INN are represented by green histograms and orange histograms respectively. Three stress values were randomly selected from the test set (we take y = 942.46,720.85, 1000.37). After that 100 sets of input parameters were generated by P-INN and INN. Since the real input parameters are not available to obtain, we use simulation to convert the reverse predicted values. In this way, the input parameters distribution is referred to by the output parameters distribution. The true stress value is marked with a red dashed line. It can be found that both P-INN and INN cover the ground truth, but the distribution of our proposed method is more concentrated, which means that our inverse process is more reliable and convincing. For a given specific stress, the solution generated by P-INN has a higher probability of manifesting the desired performance compared to the solution generated by INN. In Figure 8c and Figure 8e, although the probability of the interval in which the true value falls is not the maximum, the interval adjoins the maximum probability interval. Moreover, the truth interval probability of P-INN is larger than that of INN.

Based on the above analysis, we use P-INN to generate thorough and reliable turbine blade design solutions, given the stress conditions. Specifically, the distance between the blade and the case is specified in advance, while the stress and strain during operation are known; possible design solutions (including design parameters such as blade material) are available through P-INN as a way to avoid collisions between the blade and the chassis. Our proposed method indeed contributes to the inverse design of turbine blades.

5.   Conclusions

This work proposes the P-INN framework incorporating both epistemic and aleatory uncertainty for better inverse design and reasoning. The bi-directional loss is developed to enhance the inverse prediction accuracy. We applied the method to two mathematical problems and turbine blade design, as demonstrated by comparing the results with standard INN and DNN. Experiments show that P-INN provides not only more accurate forward predictions, but also generates more concentrated and reliable inverse posterior distributions. After training, the P-INN can generate possible inverse solutions corresponding to the target system response within a second, which is beneficial for the fast design space exploration and reasoning of critical engineering systems. Although promising results have been observed with P-INN, further study is needed regarding the down-sampling of design candidates and the screening of optimal solutions.

Acknowledgments

This work has been supported by the National Key R & D Program of China (2018YFB1700700), the National Natural Science Foundation of China (52165029), Hainan Key Research and Development program (ZDYF2021GXJS022).

Conflict of interest

The authors declare there is no conflict of interest.