An approach to solving optimal control problems of nonlinear systems by introducing detail-reward mechanism in deep reinforcement learning

1.   Introduction

Five basic quantities (voltage, charge, current, capacitance, and resistance) in electrostatics are involved in almost all applications. Electrostatics generally plays an important role in improving the performance of microelectro/mechanical systems (MEMS) and electron devices in the design stage. Many numerical methods (e.g., the finite difference method, the variational method, the moment method, the finite element method, and the boundary element method) were popularly used for engineering problems. Among diverse numerical techniques, the finite element method (FEM) and the boundary integral equation method (BIEM), as well as the boundary element method (BEM) become acceptable tools for engineers due to the increasing development of digital computing power. Here, we may focus on the mathematical study of the BIEM for electrostatics of two identical cylinders. Researchers have paid attention to the dual BEM paper of IEEE in 2003 ^[1], which has received nearly 5000 views in the Research Gate.

For a pair of two conducting cylinders, there is a large amount of literature on charged cylinders ^[2,3,4]. Different solutions existed to the electrostatic problem of two identical parallel cylinders held at the same (symmetric) potential ^[2,3]. A note was given to show their equivalence, and the identities were confirmed ^[4]. Four distinct solutions for the potential distribution around two equal circular parallel conducting cylinders by ^[2,3,5,6] were demonstrated to be equivalent by Lekner ^[7] by ways of several identities. Here, we may try an alternative way of BIEM using degenerate kernels to revisit this problem. A degenerate kernel is based on the method of separation variables, but it separates the variables in the two-point kernel function. Although the BIE in conjunction with the available degenerate kernel can only solve simple geometries and the results may be obtained more directly by using the method of separation variables for the solution instead of the fundamental solution, the tool can explain the rank-deficiency mechanism in the BIE/BEM such as degenerate scale, degenerate boundary, spurious eigenvalues and fictitious frequency, which is meaningful to the BEM community. Besides, symmetric and anti-symmetric cases are both considered. Regarding the anti-symmetric electrostatic potential, Lebedev et al. ^[8] have provided a closed-form solution by using the bipolar coordinates. The solution is interestingly found to be the simplest method of fundamental solution (MFS) of two opposite strengths of sources at the two foci. It is not trivial to check the asymptotic behavior at infinity of the two cases, symmetric and anti-symmetric. Besides, whether the equilibrium of the boundary flux along the two cylinders is satisfied or not is also our main concern.

Regarding the potential problem of a two-dimensional plane containing two circular boundaries, Chen and Shen ^[9] studied the multiply-connected Laplace problem. They found that a degenerate scale depends on the outer boundary. Chen et al. ^[10] solved the Laplace problem by using the BIEM in conjunction with the degenerate kernel to derive an analytical solution. It is found that a degenerate scale may occur due to the introduction of the logarithmic kernel for the two-dimensional case. Efficient techniques for the rank-deficiency of the BEM in electrostatic problems were proposed by Chyuan et al. ^[11]. Later, it was found that the special (degenerate) geometry happened to be the shape of unit logarithmic capacity. Kuo et al. ^[12] studied the degenerate scale for regular N-gon domains by using complex variables. Numerical implementation was also done by using the BEM. Kuo et al. ^[13] revisited the degenerate scale for an infinite plane problem containing two circular holes using the conformal mapping. Chen et al. ^[14] linked the logarithmic capacity in the potential theory and the degenerate scale in the BEM for two tangent discs. The logarithmic capacity of the line segment as well as the double degeneracy in the BIEM/BEM was studied by Chen et al. ^[15]. Due to the use of the two-dimensional fundamental solution in the BIEM, the solution space is expanded, and sometimes the corresponding matrix is rank deficient in the BEM. In other words, the integral operator of the logarithmic kernel is range deficient. A corresponding chart to show the rank deficiency and the null space of the integral operator of single, double layer potentials and their derivatives was given in ^[16,17,18], while the original one was provided in the face cover of the Strang book ^[19]. Fikioris et al. ^[20] solved rectangularly shielded lines by using the Carleman-Vekua method. In the mentioned paper ^[20], it is interesting to find that its formulation also needs a constraint ^[21] to ensure a unique solution. This outcome is similar to the paper of Chen et al. ^[22] using the Fichera's approach, where an additional constraint is also required.

In this paper, we revisit two cylinders of electrostatics by using the BIE with the degenerate kernel of the bipolar coordinates. Both the symmetric and anti-symmetric specified potentials are considered. Besides, the logarithmic capacity is also discussed. The boundary potential and flux are expanded by using the Fourier series, while the fundamental solution is represented by using the degenerate kernel. The equilibrium of boundary flux and the asymptotic behavior at infinity are also examined. The solution space expanded using the BIEM is compared with the true solution space. After summarizing the single (circle and ellipse) and two cylinders, a conclusion for constructing the solution space can be made.

2.   Electrostatic potential subject to a single cylinder

2.1.   A circular case

First, we consider a conducting cylinder. The governing equation and the Dirichlet boundary condition are shown below:

where ${\nabla }^{2}$, D and B are the Laplace operator, the domain of interest and the boundary, respectively. Furthermore, x is the position vector of a field point and $\stackrel{̄}{u}\left(\mathbf{x}\right)$ is the specified B.C. The integral formulation for the Laplace problem is derived from Green's third identity. The representation of the conventional integral equation for the domain point is written as

where s is the position vector of a source point, $U(\mathbf{s}, \mathbf{x}) = \ \mathrm{ln} \ \left|\mathbf{x}-\mathbf{s}\right| \ $ is the fundamental solution, $T(\mathbf{s}, \mathbf{x}) = \frac{\partial U(\mathbf{s}, \mathbf{x})}{\partial {\boldsymbol{n}}_{\boldsymbol{s}}}$, and $t\left(\mathbf{x}\right)$ is the unknown boundary flux. By moving the field point to the smooth boundary, Eq (2) becomes:

where the C.P.V. denotes the Cauchy principal value, and $T(\mathbf{s}, \mathbf{x}) = \partial U(\mathbf{s}, \mathbf{x})/\partial {n}_{s}$ is the closed-form kernel. Once the field point x locates outside the domain, we obtain the null-field integral equation as shown below:

where ${D}^{c}$ is the complementary domain. By employing the proper degenerate kernel ($U(\mathbf{s}, \mathbf{x})$) to represent the closed-form fundamental solution, the collocation point can be exactly located on the real boundary free of facing the singular integral. Equations (2) and (4) can be rewritten as:

and

where ${T}_{dk}(\mathbf{s}, \mathbf{x})$ and ${U}_{dk}(\mathbf{s}, \mathbf{x})$ are the corresponding degenerate kernels to represent $T(\mathbf{s}, \mathbf{x})$ and $U(\mathbf{s}, \mathbf{x})$, respectively. By setting the field point $\mathbf{x} = \left(\rho , \phi \right)$ and the source point $\mathbf{s} = \left(R, \theta \right)$ in the polar coordinates for a circular domain, the closed-form fundamental solution in Eqs (5) and (6) can be expressed by using the degenerate kernel form as shown below:

and

The unknown boundary flux $t\left(\boldsymbol{s}\right)$ is expanded in terms of Fourier series as shown below:

where ${J}_{\boldsymbol{s}} = 1$ is the Jacobian term, ${a}_{0}$, ${a}_{n}$ and ${b}_{n}$ are unknown coefficients. The given boundary condition is

where v is a constant. By considering $R = a$ in Eqs (6)–(8), the coefficient of the Fourier constant base is

where $a$ is the radius of the circular cylinder. Equation (11) indicates that the occurring mechanism of a degenerate scale is

When $a = 1$, the coefficient of ${a}_{0}$ cannot be determined. It results in a non-unique solution. This critical size is called a degenerate scale. In Rumely's book ^[23], the logarithmic capacity, ${c}_{L}$, of a circle is equal to its radius. It is easily found that the special (degenerate) geometry happens to be the shape of unit logarithmic capacity. The discriminant ${D}_{p}\left(a\right)$ of the degenerate scale in the BEM/BIEM for a circular boundary is written as

If ${D}_{p}\left(a\right)\ne 0$, this size is an ordinary scale and there exists a unique solution. Otherwise, according to the Fredholm alternative theorem, there is no solution or infinite solutions. For an ordinary scale, the boundary flux, of the electrostatic field along the boundary, is obtained as

The unique solution of electrostatic potential is obtained by

as shown in Figure 1(a). Even though the electrostatic field along the boundary in Eq (14) is not in equilibrium, i.e. ${\int }_{B}t\left(\mathbf{x}\right)dB\left(\mathbf{x}\right)\ne 0$, the electrostatic field at infinity, ${\Gamma }^{\infty }$, would exist and satisfy the equilibrium condition together in total, ${\int }_{B+{\Gamma }^{\infty }}t\left(\mathbf{x}\right)dB\left(\mathbf{x}\right) = 0$. If we normalize the potential on the cylinder to the unity, and let λ be the dimensionless ratio, the potential becomes

where

The solution by using the direct BIE of Eq (15) is the special case of Eq (16) by setting $\lambda = \frac{v}{{D}_{p}\left(a\right)}$. When the size of the boundary is a degenerate scale, i.e., $a = 1$ and ${D}_{p}\left(a\right) = \ \mathrm{ln} \ a = 0$, it has no solution if $v\ne 0$. If $v = 0$, then the constant term in Eq (9), ${a}_{0}$, is a free constant. The electrostatic potential yields

and Eq (16) would reduce to

and ${u}_{d}\left(\mathbf{x}\right)$ in Eq (19) reduces to $\ \mathrm{ln} \ \rho$ since $\ \mathrm{ln} \ a = 0$. It is easy to find that ${a}_{0}$ and $\lambda$ are equivalent.

2.2.   An elliptical case

For an elliptical case, we naturally utilize the elliptic coordinates to solve the problem in the BIE. The relation between the Cartesian coordinates and the elliptic coordinates is given below:

where c is the focal length. By separating the source point and the field point in the elliptic coordinates ^[24] to represent the closed-form fundamental solution, we have

where ${J}_{s} = c\sqrt{{ \ \mathrm{cosh} \ }^{2}{\xi }_{s}{ \ \mathrm{sin} \ }^{2}{\eta }_{s}+{ \ \mathrm{sinh} \ }^{2}{\xi }_{s}{ \ \mathrm{cos} \ }^{2}{\eta }_{s}}$. The unknown boundary flux $t\left(\boldsymbol{s}\right)$ is expanded in terms of generalized Fourier series. We have

where ${a_0}$, ${a_n}$ and ${b_n}$ are unknown coefficients. The given boundary condition is

By substituting Eqs (21a), (22a), (23) and (24) into Eq (6), the coefficient of the Fourier constant base is

Equation (25) indicates that the occurring mechanism of a degenerate scale is

Equation (26) yields the degenerate scale of $\frac{a+b}{2} = 1$, where a and b are the semi-major and semi-minor axes of an ellipse, respectively. According to Eq (25), the discriminant of a degenerate scale in the BEM/BIEM is obtained

In Rumely's book ^[23], the logarithmic capacity of an ellipse is equal to $\frac{a+b}{2}$. According to Eqs (13) and (27), the logarithmic capacity, ${c}_{L}$, and the discriminant, ${D}_{e}(\cdot )$, satisfy the relation,

The relationship of the discriminant, logarithmic capacity, and degenerate scale are summarized in Table 1. If ${D}_{e}(c, {\xi }_{0})$ is not equal to zero, this size is an ordinary scale with a unique solution. Otherwise, according to the Fredholm alternative theorem, it has no solution or infinite solution. For an ordinary scale, the boundary flux is obtained by

The unique solution of electrostatic potential is

as shown in Figure 1(b). Even though the boundary flux in Eq (29) is not in equilibrium, i.e., ${\int }_{B}t\left(\mathbf{x}\right)dB\left(\mathbf{x}\right)\ne 0$, the electrostatic field at infinity, ${\Gamma }^{\infty }$, would exist and satisfy the equilibrium condition together in total, i.e., ${\int }_{B+{\Gamma }^{\infty }}t\left(\mathbf{x}\right)dB\left(\mathbf{x}\right) = 0$. If we normalize the potential on the cylinder to the unity, and let λ be the dimensionless ratio, the potential becomes

where

A neat formula of ${u}_{d}\left(\mathbf{x}\right)$ could be defined as ${u}_{d}\left(\mathbf{x}\right) = {D}_{e}\left({\xi }_{\boldsymbol{x}}\right)-{D}_{e}\left({\xi }_{0}\right)$. The solution by using the direct BIE of Eq (30) is the special case of Eq (31), if $\lambda = \frac{v}{{D}_{e}(c, {\xi }_{0})}$.

When the size of the boundary is a degenerate scale, ${D}_{e}(c, {\xi }_{0}) = {\xi }_{0}+ {ln}\frac{c}{2} = 0$, it is no solution if $v\ne 0$. If $v = 0$, the constant term in Eq (23), ${a}_{0}$, is a free constant. The electrostatic potential yields

and Eq (16) reduces to

where ${u}_{d}\left(\mathbf{x}\right)$ in Eq (32) reduces to ${\xi }_{\boldsymbol{x}}+ \ \mathrm{ln} \ \frac{c}{2}$ since ${D}_{e}\left({\xi }_{0}\right)$ is equal to zero. It is easy to find that ${a}_{0}$ and $\lambda$ are equivalent. In addition, the degenerate scale in the BEM/BIEM is due to the logarithmic kernel.

For the single elliptical cylinder, the degenerate kernel is expanded in terms of the generalized form as

where $\xi$ and $\eta$ are the radial and angular directions, respectively, ${D}_{e}(\cdot )$ is the constant function for ${\eta }_{\boldsymbol{s}}$ and other term is $\alpha (\cdot )$. The unknown boundary flux $t\left(\mathbf{s}\right)$ is expanded in terms of the generalized Fourier series as shown below:

where ${a}_{0}$, ${a}_{n}$ and ${b}_{n}$ are unknown coefficients. By substituting Eqs (35a), (36a), (37) and the boundary condition (Eq (10)) into Eq (6), the coefficient of the Fourier constant base is

If $D({\xi _0}) \ne 0$, then the unique solution of electrostatic potential is

and

When the size of the boundary is a degenerate scale, ${D}_{e}\left({\xi }_{0}\right) = 0$, there is no solution if $v\ne 0$. If $v = 0$, the constant term in Eq (37), ${a}_{0}$, is a free constant. The electrostatic potential yields

and Eq (40) reduces to

where ${u}_{d}\left(\mathbf{x}\right)$ in Eq (40) is reduced to ${D}_{e}\left({\xi }_{x}\right)$, since ${D}_{e}\left({\xi }_{0}\right)$ is equal to zero. By using the generalized form of Eqs (35) and (36), the analytical and neat form of ${u}_{d}\left(\mathbf{x}\right)$ for the single elliptical cylinder is derived. It is easy to find that ${a}_{0}$ and $\lambda$ are equivalent. The generalized potential and the solution by using the BIEM are compared in Table 2.

3.   Problems of two identical cylinders in electrostatics: symmetric and anti-symmetric cases

In this section, we consider two circular cylinders of electrostatics. The Dirichlet boundary conditions of two circular cylinders are given by

where ${u}^{l}\left(\mathbf{x}\right) \ \text{and} \ {u}^{r}\left(\mathbf{x}\right)$ are potentials of the left and right circular boundaries, respectively, x is the position vector of the field point, B is the boundary, and ${v}_{1} \ \text{and} \ {v}_{2}$ are specified constant potentials. The original problem can be decomposed into a symmetric problem and an anti-symmetric problem as shown below:

and

Since the problem contains two circular boundaries, we naturally employ the bipolar coordinates to express the closed-form fundamental solution. The relation between the Cartesian coordinates and the bipolar coordinates is shown below:

where $\eta$ and $\xi$ are the radial and angular coordinates, respectively, c is the half distance between the two foci of the bipolar coordinates. By separating the source point and the field point in the bipolar coordinates ^[9] for the closed-form fundamental solution, we have

where $\mathbf{x} = ({\eta }_{x}, {\xi }_{x})$, $\mathbf{s} = ({\eta }_{s}, {\xi }_{s})$ and ${J}_{s} = c/\left[{cos}h({\eta }_{s})- {cos}({\xi }_{s})\right]$.

3.1.   Derivation of the analytical solution for the symmetry problem

The boundary condition of the symmetry problem is shown in Eq (44). The unknown boundary densities on the two circular cylinders can be expanded by using the generalized Fourier series as shown below:

where ${a}_{0}^{l}, {a}_{n}^{l}, {b}_{n}^{l}, {a}_{0}^{r}, {a}_{n}^{r}$ and ${b}_{n}^{r}$ are unknown coefficients of the generalized Fourier series. By substituting Eqs (47a), (47f), (48a), (48f), (44) and (49) into Eq (6), and collocating the null-field point on the left boundary, ${B}^{l}$, we have

Similarly, substituting Eqs (47c), (47d), (48c), (48d), (44) and (49) into Eq (6), and collocating the null-field point on the right boundary, ${B}^{r}$, we have

By adding Eqs (50) and (51) together, we obtain

After comparing the coefficient of generalized Fourier bases, we have

By similarly subtracting Eq (50) from Eq (51), we have

After comparing the coefficient of generalized Fourier bases, we have

In order to solve the coefficients ${a}_{0}^{l}$ and ${a}_{0}^{r}$, we need to define a discriminant as shown below:

For the case of two cylinders, Rumely ^[23] employed the complex variable to derive the logarithmic capacity, as shown in Table 1. Since the logarithmic capacity is not a closed-form or an exact formula, the postulate in Eq (28) for the case of two cylinders could not be analytically verified at present. If ${D}_{b}\ne 0$, the geometry of the problem is an ordinary scale, Eqs (53) and (55) yield the coefficients as shown below:

Substituting Eqs (47b), (47f), (48b), (48f), (44) and the obtained unknown boundary densities into Eq (5) for the field solution of ${\eta }_{x} < 0$, we have the unique solution

Similar substitution of Eqs (47c), (47e), (48c), (48e), (44) and the obtained unknown densities into Eq (5), the field solution for ${\eta }_{x}\ge 0$ yields

It is found that Eqs (58) and (59) show the symmetry solution. All potentials are shown in Figure 1(c). This solution will be compared and discussed with that of Darevski ^[2] later.

If ${D}_{b}(c, {\eta }_{0}) = 0$, a degenerate scale occurs. When the constant potential $v\ne 0$, it yields no solution. When the constant potential $v = 0$, it yields infinite solutions. Equations (53) and (55) yield the coefficients as shown below:

where k is an arbitrary constant. In case of a degenerate scale, ${\eta }_{0}$ becomes

Substituting Eqs (47b), (47f), (48b), (48f), (44) and the obtained boundary unknown densities into Eq (5) for the field solution of ${\eta }_{x} < 0$, we have the infinite solution,

Similar substitution of Eqs (47c), (47e), (48c), (48e), (44) and the obtained boundary unknown densities into Eq (5), the field solution for ${\eta }_{x}\ge 0$ yields the infinite solution,

Equations (62) and (63) also indicate symmetry.

3.2.   Derivation of the analytical solution for the anti-symmetry problem

Similarly, we consider the anti-symmetry problem. The coefficient of generalized Fourier bases in Eqs (45) and (49) satisfy

and

We also find the discriminant, ${D}_{b}(c, {\eta }_{0})$ in Eq (64). If ${D}_{b}(c, {\eta }_{0})\ne 0$, the geometry of the problem is an ordinary scale. Equations (64) and (65) yield the coefficients as shown below:

Substituting Eqs (47b), (47f), (48b), (48f), (45) and the obtained boundary unknown densities into Eq (5) for the field solution of ${\eta }_{x} < 0$, we have

Substituting Eqs (47c), (47e), (48c), (48e), (45) and the obtained unknown densities into Eq (5) for the field solution of ${\eta }_{x}\ge 0$, we also have

All potentials are shown in Figure 1(d). The solution in Eq (68) matches well with that of Lebedev et al. ^[8]. From the viewpoint of the MFS, this solution is the simplest one since only two sources with opposite strengths are required to locate the two foci.

If ${D}_{b}(c, {\eta }_{0}) = 0$, a degenerate scale occurs. Fortunately, it doesn't result in no solution whether v is equal to zero or not as shown in the boundary condition of Eq (45). Equations (64) and (65) yield the coefficients as shown below:

where k is an arbitrary constant. For a degenerate scale case, ${\eta }_{0}$ satisfies ${D}_{b}(c, {\eta }_{0}) = 0$, i.e.

Substituting Eqs (47b), (47f), (48b), (48f), (45) and the obtained boundary unknown densities into Eq (5) for the field solution of ${\eta }_{x} < 0$, we have

Similarly substituting Eqs (47c), (47e), (48c), (48e), (45) and the obtained unknown densities into Eq (5), we obtain the field solution

Equations (71) and (72) destroy the anti-symmetry due to the second part of k. To obey the anti-symmetry solution, k should be zero. In other words, this k part in the solution of Eqs (71) and (72) also disobey the bounded potential at infinity. This solution for a free constant, k, will be compared with that of Lekner ^[4] later.

3.3.   Results and discussions

According to the solution of Lekner ^[4], the general solution space of the symmetry problem in Eq (44) is expressed as follows:

where

By using the identity equation,

the solution by using the direct BIE of Eq (59) is rewritten as

Equation (59) is the special case of Eq (73), if $\lambda = \frac{-v}{{D}_{b}(c, {\eta }_{0})}$. When the size of the boundary is a degenerate scale, i.e., ${D}_{b}(c, {\eta }_{0}) = 0$, the BIE solution does not exist if $v\ne 0$. If $v = 0$, the constant term in Eq (57), ${a}_{0}^{r}$, is a free constant. The electrostatic potential is obtained by

and Eq (73) can be reduced to

since v is zero. It is easy to find that ${a}_{0}^{r}$ and $\lambda$ are equivalent.

Similarly, the general solution space of the anti-symmetry problem in Eq (45) is expressed as follows:

where

Lebedev et al. ^[8] considered the condition at the infinity, $u\left(\boldsymbol{x}\right) = 0, \boldsymbol{x}\to \infty$, the solution of Eq (79) would reduce to only

It is the reason why the solution of Eq (68) by using the BIEM is a special case of Eq (79) for $\lambda = 0$. When the size of the boundary is a degenerate scale, i.e., ${D}_{b}(c, {\eta }_{0}) = 0$, it yields infinite solutions. Since the sum of constant terms, ${a}_{0}^{r}$ and ${a}_{0}^{l}$, in Eq (69) is a free constant, $k$, the electrostatic potential yields

It is easy to find that $k$ and $\lambda$ are equivalent. To sum up, the free constant, $\lambda$ and ${u}_{d}\left(\mathbf{x}\right)$ in the general solution by Lekner ^[4] are similar to the constant term in the boundary potential and the obtained BIE solution for the degenerate scale by Chen et al. ^[9], respectively. The obtained BIE solution for the degenerate case yields nontrivial boundary flux even though the boundary potential is trivial. The generalized potential and the available solutions by using the BIEM for the problem containing two cylinders are compared with each other in Table 3.

4.   Conclusions

This paper investigates the solution space for the electrostatics of two cylinders using the BIEM. Both the symmetric and anti-symmetric cases are considered. Flux equilibrium on the cylindrical boundaries and the asymptotic behavior at infinity is also examined. Moreover, on the base of the Fredholm alternative theorem, the relation of unique solution and the degenerate scale in the BIEM is linked. The logarithmic capacity and the discriminant are also linked by using an exponential relation. Besides, the degenerate scale is also related. Not only two cylinders but also a single one (circle or ellipse) are considered. Finally, the results are compared with those derived by other researchers. Linkage and agreement are made.

Acknowledgments

The authors wish to thank the financial supports from the National Science and Technology Council, Taiwan under Grant No. MOST 111-2221-E-019-009-MY3 for National Taiwan Ocean University.

Conflict of interest

The authors declare there is no conflict of interest.