Neuro-Swarm heuristic using interior-point algorithm to solve a third kind of multi-singular nonlinear system

1.   Introduction

The historical Lane-Emden model was introduced first time by astrophysicist Jonathan Homer Lane and Robert Emden ^[1,2] working on the thermal performance of a spherical cloud of gas and classical law linked to thermodynamics ^[3]. The singular models have several applications in broad field of applied science and engineering such as catalytic diffusion reactions along with error estimate problems ^[4], density profile of gaseous star ^[5], stellar configuration ^[6], spherical annulus ^[7], isotropic continuous media ^[8], the theory of electromagnetic ^[9] and morphogenesis ^[10]. It is always not easy to solve the system of singular equations-based models due to their complex nature and singular points. To mention a few schemes that have been applied to solve such models include Legendre wavelets spectral technique ^[11], Bernoulli collocation scheme ^[12], variational iteration technique ^[13], Haar wavelet quasilinearization method ^[14], spectral collocation scheme ^[15], differential transformation approach ^[16] and Adomian decomposition technique ^[17].

All these above cited approaches have their precise merits and imperfections, however stochastic solver has not been extensively implemented to solve multi-singular third kind of nonlinear system (MS-TKNS) using the artificial neural networks (ANNs) together with particle swarm optimization (PSO) and interior-point algorithm (IPA), i.e., ANN-PSO-IPA. The stochastic computing solvers have been widely applied to resolve numerous applications ^{[18,19,20,21,22]}. Recently, the stochastic solvers presented the solution of models for financial market forecasting ^[23], prey-predator nonlinear system ^[24], nonlinear singular functional differential model ^[25,26], SITR nonlinear system ^[27,28], singular delay differential system ^[29], nonlinear periodic boundary value problems ^[30], HIV nonlinear system ^[31], SIR nonlinear system of dengue fever ^[32] and alternative approach based on fuzzy-neuro methods to solve linear and nonlinear optimization problems ^[33,34]. These submissions enhance the worth of the stochastic solvers to authenticate the convergence and precision of the suggested ANN-PSO-IPA. The general form of the MS-TKNS is written as ^[35]:

Where ${F}_{1}$ and ${F}_{2}$ are the nonlinear functions, $P$ and $Q$ are positive constants, ${G}_{1}$ and ${G}_{2}$ are indicated as a source functions.

The purpose of this study is to present the solution of the model (1) via intelligent computing ANN-PSO-IPA. The contributions of the paper are as follows:

●  A novel neuro-swarm computing intelligent heuristics ANN-PSO-IPA is accessible for multi-singular nonlinear third order EF-SDEs.

●  The overlapping outcomes of the proposed ANN-PSO-IPA with the exact outcomes for three examples of MS-TKNS enhance the exactness, consistency and convergence.

●  Authorization of the precise performance is validated via statistical remark using the ANN-PSO-IPA based on Theil's Inequality Coefficient (TIC), Root Mean Square Error (RMSE), Variance Account For (VAF), Semi Interquartile (SI) Range.

●  Beside practically precise continuous outcomes on whole input training intermission, an easy implementable process, simplicity in perception, stability and robustness are other well-intentioned announcements for the designed neuro-swarm intelligent computing approach.

The remaining structure of the current study is given as: Section 2 indicates the design methodology through PSO-IPA. The mathematical form of performance measures can be found in section 3. Section 4 shows the numerical results of the designed ANN-PSO-IPA. In the final section, final submissions and future guidance are provided.

2.   Methodology

The design of ANN-PSO-IPA for MS-TKNS is presented in two steps, given as:

Step 1: An error based objective function is accessible by using the mean square error sense.

Step 2: The learning procedure of the structures is presented using the hybrid of PSO-IPA.

2.1.   ANN modeling

The ANNs are famous to solve the various applications in different domain of applied science and engineering. The proposed outcomes are denoted by $U\left(\chi \right)$ and $V\left(\chi \right)$, while $\frac{{d}^{n}U}{d{\chi }^{n}}$ and $\frac{{d}^{n}V}{d{\chi }^{n}}$ show the n^th derivative, mathematically given as:

Where m shows the neurons and n is the derivative order. The unknown weight vectors are $a, w$ and $\phi$. $W = [{W}_{U}, {W}_{V}]$, for ${W}_{U} = [{\alpha }_{U}, {w}_{U}, {\phi }_{U}]$ and ${W}_{V} = [{\alpha }_{V}, {w}_{V}, {\phi }_{V}]$. The components of the weight vector are given as:

The log-sigmoid $Z\left(\chi \right) = \frac{1}{\left(1+{e}^{-\chi }\right)}$ is used as an activation function. The updated form of the system (2) using the approximate results of $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$ are written as:

The error based objective formulation is written as:

Where $N = \frac{1}{h}, {x}_{m} = mh$. The Objective functions ${E}_{1}$ and ${E}_{2}$ are associated with the system of differential equations and ${E}_{3}$ is the corresponding initial conditions.

2.2.   Optimization procedure

The optimization is performed to solve the MS-TKNS using the hybrid framework of PSO- IPA.

PSO is an effective research method that has widely used as an alternative optimization of genetic algorithms that were discovered by Kennedy and Eberhart ^[36]. In the theory of search space, a single candidate result of decision variables in the optimization procedure is called a particle and set of these particle formulated a swarms. For the refinement of optimization variables in standard PSO utilized iterative process of optimizing based on local ${P}_{LB}^{\rho -1}$ and global ${P}_{GB}^{\rho -1}$ best position of the particle in the swarm. The mathematical relations of position X_i along with the velocity V_i in PSO are given, respectively, as follows:

where $\rho$represent the current flight index, the inertia vector is denoted by $\omega$ varying between 0 and 1, ${\eta }_{1}$ and ${\eta }_{2}$ indicate the cognitive and social accelerations, respectively, while, r₁ and r₂ are vectors form with pseudo real number between 0 and 1. Further information regrading PSO can be seen in ^[37], while few recent applications address by PSO include parameter estimation ^[38], nonlinear electric circuits ^[39], optimize performance of induction generator ^[40], optimization of permanent magnets synchronous motor ^[41] and systems of equations based physical models ^[42].

The quickly converges performance of PSO is attained by the process of hybridization with the appropriate local search approach by taking the PSO best values as an initial weight. Consequently, in the presented study, an effective local search scheme based on interior-point (IPA) is exploited for rapid fine-tuning of the results by the PSO algorithm. The hybrid of PSO-IPA train the ANNs as well as fundamental parameter setting for both PSO and IPA are tabulated in Table 1. Recently, IPA is used to power flow optimization incorporating security constraints ^[43], multistage nonlinear nonconvex problems ^[44], image processing ^[45] and multi-fractional order doubly singular model ^[46]. The hybrid of PSO-IPA train the decision variables of ANNs as per procedure and settings tabulated in Table 1.

3.   Performance indices

The current study is associated to present the statistical measures for solving the MS-TKNS. Therefore, three performances based on Theil's inequality coefficient (TIC) mean absolute deviation (MAD) and Variance Account For (VAF) and their global variables are Global TIC (G.TIC), Global MAD (G.MAD) and Global EVAF (G.EVAF) are applied. The mathematical descriptions of these statistical operators are provided as:

4.   Results and discussion

In this section, the detail discussion to solve three variants of the MS-TKNS is presented.

Problem 1: Consider the MS-TKNS is:

The exact/true solutions of the above Eq (13) are [${e}^{{\chi }^{3}}, {e}^{-{\chi }^{3}}$] and the fitness function becomes as:

Problem 2: Consider the MS-TKNS is:

The true solutions of the above equation are [$1+{e}^{{\chi }^{3}}, 1+{e}^{-{\chi }^{3}}$] and the objective function becomes as:

Problem 3: Consider the MS-TKNS is:

The true solutions of the Eq (17) are [$1+{\chi }^{3}, 1-{\chi }^{3}$] and the fitness function becomes as:

In order to find the proposed solutions of the Problems 1, 2 and 3 based on the MS-TKNS by using the proposed solver ANN-PSO-IPA for 40 multiple trials to achieve the adaptable parameters. The plots of the weight sets are shown in Figure 1 for $U\left(\chi \right)$ and $V\left(\chi \right)$, respectively. These weights are the decision variables of ANNs as presented in equations 3 such that the fitness functions in (14), (16) and (18) for respective problems 1, 2 and 3 are optimized with PSO-IPA, i.e., initially for global search efficacy of PSO and fine tune with IPA for rapid local search. These sets of weights are applied in first equation of set (3) to find approximate solutions to the three problems and the mathematical form are given as:

The optimization of the MS-TKNS is performed for the problems 1, 2 and 3 using the proposed solver ANN-PSO-IPA for 40 independent trials. Set of weights and results comparison are plotted graphically in Figure 1. It is specified that the exact and proposed solutions overlapped for both the indexes $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$ of the problems 1, 2 and 3. This exact matches of the outcomes shows the correctness of the proposed methodology ANN-PSO-IPA. In order to calculate the comparison of the numerical results, the plots of the absolute error (AE) are drawn in Figure 2(a), (b) for $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$. One can observe that most of the AE values of problems 1–3 for $\widehat{U}\left(\chi \right)$ lie in the range of 10⁻⁶ to 10⁻⁷, 10⁻⁴ to 10⁻⁶ and 10⁻⁶ to 10⁻⁸, while, for $\widehat{V}\left(\chi \right)$, these values lie around 10⁻⁵ to 10⁻⁶, 10⁻³ to 10⁻⁴ and 10⁻⁶ to 10⁻⁷. The plots of the performance measures through fitness, RMSE, TIC and EVAF are drawn in the Figure 2(c), (d) for $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$. It is seen that the fitness values lie around to 10⁻⁰⁶ to 10_,⁻⁰⁸ for problems 1 and 3, while the fitness values for Problem 2 are close to 10⁻⁰⁸. The RMSE values of $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$ for problem 1 and 3 are close to 10⁻⁶ to 10⁻⁸, while for Problem 2, the RMSE lie 10⁻⁴–10⁻⁶ for $\widehat{U}\left(\chi \right)$ and 10⁻²–10^–4 for $\widehat{V}\left(\chi \right)$. The TIC performance lie around 10⁻⁶–10⁻⁸ for both indexes of all the Problems. The EVAF values for problem 1 and 3 lie 10⁻¹⁰–10⁻¹² for both the indexes, while it lies around 10⁻⁸–10⁻¹⁰ for problem 2.

The convergence measures for the Problems 1–3 based on the MS-TKNS using the Fitness, histograms and boxplots for 10 numbers of neurons are provided in Figure 3. It is shown that most of the fitness values lie around 10⁻⁴–10⁻⁶ for Problem 1 and 3, while for Problem 3 these values lie around 10⁻⁶–10⁻⁸. The convergence of both the indexes of all the problems for RMSE, TIC and EVAF is provided in Figures 4–9. Most of the values for both the indexes of all the problems lie in good ranges.

For more accuracy and precision, statistical indices are performed based on minimum (Min), standard deviation (SD), mean, SI range and Median. SI Range is one half of the difference of Q₃ = 75% data, i.e., 3^rd quartile and Q₁ = 25% data, i.e., 1^st quartile is calculated for 40 trials of ANN-PSO-IPA to solve three different problems of MS-TKNS. These statistical based outcomes for Problems 1–3 are tabulated in Tables 2 and 3 for the indexes $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$. It is observed that most of the $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$ values for Problems 1–3 lie in the best ranges. The global performance G.FIT, G.RMSE, G.TIC and G.EVAF of $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$ for Problems 1, 2 and 3 are tabulated in Table 4. These performances of the global values for Problems 1, 2 and 3 for 40 independent trials are provided. The magnitude (Mag) and Median values for all Problems using the indexes $\widehat{U}\left(\chi \right)$ and $\widehat{V}\left(\chi \right)$ proved very good results based on the statistical global operators.

5.   Conclusions

In this research study, a stable, reliable and accurate numerical ANN-PSO-IPA is accessible to solve the multi-singular nonlinear third kind of Emden-Fowler system by using the ANN strength with continuous mapping. A fitness function of these networks is optimized for the global and local search capabilities of particle swarm optimization and interior-point algorithm, respectively. The proposed ANN-PSO-IPA is broadly applied to solve three different variants of the multi-singular nonlinear third kind of Emden-Fowler system. The precise and accurate performance is observed for ANN-PSO-IPA based on AE with consistent precision around 5 to 8 decimal places of precision for all three problems of the multi-singular nonlinear third kind of Emden-Fowler system. Statistical interpretations in terms of Min, Mean, SD, SI ranges and Median are performed to validate the convergence, robustness and accuracy of the proposed ANN-PSO-IPA for solving the multi-singular nonlinear third kind of Emden-Fowler system based Eqs 1–3.

In the future, new stochastic solvers based on ANN optimized with evolutionary/swarming paradigm looks proficient to solve nonlinear biological systems ^{[47,48,49,50]}, fluid dynamics models ^{[51,52,53,54,55,56]} and fractional models ^{[57,58,59,60]}. Additionally, the different ANNs structure exploiting variety of activation functions should be implemented to solve the MS-TKNS for improved performance.

Acknowledgments

This paper has been partially supported by Ministerio de Ciencia, Innovacion y Universidades grant number PGC2018-0971-B-100 and Fundacion Seneca de la Region de Murcia grant number 20783/PI/18.

Conflict of interest

All the authors of the manuscript declared that there are no potential conflicts of interest.