Integrative genomic analysis of a novel small nucleolar RNAs prognostic signature in patients with acute myelocytic leukemia

1.   Introduction

Since McCulloch and Pitts first proposed the primary theory and learning model of artificial neural networks ^[1]. ANNs can be considered as an effective model that can get desirable results for processing supervised/unsupervised machine learning tasks ^[2]. Hence, ANNs have been widely used in various fields as a powerful and practical tool to solve real problems, such as forecast and estimation ^[3], biology and medicine ^[4], classification ^[5], pattern recognition ^[6] and so on ^[7]. Among different artificial neural networks, feed-forward neural networks (FNNs) ^[8], especially two-layer FNNs, is widely applied ^[9]. In fact, FNNs with two layers is the simplest and most widespread practical application. For any type of neural network, learning is considered an important and critical process and attracts lots of researchers. The learning process is a key field and the purpose of which is to optimize the cost function for feedforward neural networks. The cost function is considered to be the mean square error (MSE) obtained by the optimal connection of the weights and bias values.

Generally, the training methods of feedforward neural networks can be seen in two classes: gradient-based search methods and metaheuristic search methods. The most popular search method in this area is the so-called Back-Propagation (BP) algorithm ^[10], which is a gradient-based method. When dealing with complex optimal problems, it may quickly converge to the local optimal solution instead of the global optimal solution. As a result, back-propagation cannot obtain a better candidate solution. In addition, the initial solutions of weights, biases, and learning rates determines the performance of the BP algorithm. Even if the learning target is reached, this results in the slow performance of the BP algorithm.

In recent years, a growing number of scholars' regard metaheuristic search methods as the training method of feedforward neural networks to alleviate this main drawback. In the stochastic algorithms, Montana and Davis proposed genetic algorithm (GA) is the first training method to optimize the feedforward neural network ^[11], they describe a set of experiments performed on data from a sonar image classification problem. The experimental results display that GA is superior to the BP algorithm when solving complex real problems. Li et al. proposed an improving particle swarm optimization algorithm based on neighborhood and historical memory (PSONHM) ^[12], and employed to investigate the efficiencies of PSONHM in training Multi-Layer Perceptron (MLP). The experimental results indicate that the proposed PSONHM can effectively solve the global optimization problems. Gambhir et al. developed a PSO-ANN based diagnostic model for earlier diagnosis of dengue fever ^[13]. In the proposed model, PSO technique is applied to optimize the weight and bias parameters of ANN method. The effectiveness of the proposed model is evaluated based on accuracy, sensitivity, specificity, error rate and AUC parameters. The results of the proposed model have been compared with other existing approaches like ANN, DT, NB, and PSO. Mirjalili et al. proposed using a social spider optimization algorithm to train feedforward neural networks ^[14]. The experimental results show that the algorithm has fast convergence speed and high accuracy in most test data sets. Uzlu et al. study is associated with predicting energy consumption in Turkey ^[15]. GDP (gross domestic product), population, import and export were used as predictor variables. TLBO (teaching–learning-based optimization) and BP were used to train ANNs. ANN-TLBO predicted the energy consumption more accurately than ANN–BP. Using the ANN–TLBO model, the energy consumption was forecasted until 2020. Aljarah et al. proposed a new training algorithm based on the recently proposed whale optimization algorithm (WOA) ^[16]. The results are verified by comparisons with back-propagation algorithm and six evolutionary techniques. Kowalski and Łukasik to compare the Krill Herd Algorithm (KHA) optimization algorithm used for learning an artificial neural network (ANN), with other heuristic methods and with more conventional procedures ^[17]. The proposed ANN training method has been verified for the classification task. It has been concluded that the application of KHA offers promising performance-both in terms of aforementioned metrics, as well as time needed for ANN training. Alboaneen et al. used to glowworm swarm optimisation (GSO) algorithm training the MLP neural network ^[18]. The GSO based trainer is evaluated on five classification datasets, the results show that our proposed trainer achieves better classification accuracy rate in most datasets compared to the other algorithms. Heidari et al proposed a new hybrid stochastic training algorithm using the recently proposed grasshopper optimization algorithm (GOA) for multilayer perceptrons (MLPs) neural networks ^[19]. The proposed GOAMLP model is then applied to five important datasets: breast cancer, parkinson, diabetes, coronary heart disease, and orthopedic patients. It is shown and proved that the proposed stochastic training algorithm GOAMLP is substantially beneficial in improving the classification rate of MLPs. Ehsan, et al. proposed an improved cuckoo search algorithm ^[20]. It is employed for training feedforward neural networks for two benchmark classification problems. Socha and Blum proposed a Ant Colony Optimization (ACO) variant for continuous optimization ^[21,22]. The results show that the best of our algorithms are comparable to gradient-based algorithms for neural network training ^{[23,24,25,26]}.

In 2016, Mirjalili and Lewis proposed the whale optimization algorithm ^[27]. It has proved that the WOA algorithm is more competitive than other famous meta-heuristic algorithms. When dealing with the real problem, it may quickly reach to the local optimal solution instead of the global optimal solution. This phenomenon, called immature convergence, is a common problem in metaheuristics ^[28]. Another question called the stagnation in local optimal solutions ^[29]. This question can seriously lead to the degradation of the candidate solution quality. The phenomenon can happen when algorithms fail to get better between exploration and exploitation. Exploration defined as the ability of algorithm to find an unknown area in search space. The exploitation stage must follow the exploration phase. In this stage, the field of the better-explored positions in various regions is included again to get a better candidate solution ^[30]. Widespread exploration in early iterations and then focused exploitation in the final steps of optimization can help the optimizer to avert local solutions ^[31].

Studies have shown that some algorithms have better local search capabilities ^[32], some algorithms have better global search capabilities, and some optimization algorithms profit from a better balance between exploration and exploitation. Hence, hybrid algorithms can decrease the inherent defects of the original algorithm ^[33]. With this strategy, the hybrid algorithms gain all the superiority of the original algorithm. The primary intention of hybrid optimizers is to decrease the amount of computation, increase the accuracy of the results, enhanced the stability of the algorithm and the convergence trend of the basic algorithm ^[34]. This paper presents an improved whale optimization algorithm which aimed to enhance the precision of the convergence of the basic WOA algorithm. In order to test the performance of the algorithm, the optimal weight and biases of MLP neural networks are used for classification problems and experimental results indicate that this algorithm has better classification accuracy, convergence speed, and convergence precision than other algorithms in optimizing multi-layer perceptron neural network.

The research content of this paper is as follows: the basic definition of MLP is described in section 2. Section 3 describes the original WOA algorithm. The detailed description of the TSWOA is presented in section 4. Section 5 details how the TSWOA can be used for training MLP. Various experimental results and analyses were conducted in section 6. We conclude in section 7.

2.   Feedforward neural network and multi-layer perceptron

Feedforward neural network is a prevalent structure of neural network model which employs its sophisticated parallel layered to understand and approximate computing models ^[35]. It is made up of many neurons distributed in different levels. The input layer is located in the first layer of feedforward neural network, the output layer is located in the last layer, and the hidden layer is located in the other layer. Feedforward neural networks that have many hidden layers are called multilayer perceptron (MLP), which is displayed in Figure 1. In MLP, neurons are organized in a unidirectional pattern. In the network form, the message is spread in one direction, respectively through the input layer, the hidden layer, and the output layer. As shown in Figure 1, the data is input by the input layer, multiplied by the respective weights, and then taken as the input of the hidden layer. After the calculation of the hidden layer, the data is spread to the output layer by multiplying the respective weights, and the output is calculated by the output layer. The actual output of the neuron is calculated by the activation function. The operation step of MLP is displayed in Eqs (1)–(4).

Firstly, the input value of the hidden layer is obtained by using the Eq (1)

where $n$ represents the total number of the input layer, ${w_{i, j}}$ is the connection weight, bias value of the $j$-th node is expressed as $\theta {}_j^{}$, and ${x_i}$ is $i$-th input data.

The output value of the hidden layer is obtained by using the Eq (2),

Finally, the result of the output layer is described as follows:

where ${w_{j, k}}$ is the connection weight from the $j$-th node to the $k$-th node, ${\theta '_k}$ expresses the bias value of the $k$-th node in the output layer, and ${O_k}$ is the output value of the $k$-th node. From Eqs (1)–(4), it can be concluded that the connection weights and bias values are the most important parts of the MLP, and they confirm the final value of the output. The ultimate goal of optimizing MLP is to get the optimal connection weights and bias to achieve the output.

3.   Whale Optimization Algorithm (WOA)

There are three main stages in the whale optimization algorithm, which are searching prey, encircling prey and Logarithmic spiral prey. Among them, searching for prey is the exploration stage of the algorithm. Moreover, encircling prey and Logarithmic spiral prey is the exploitation stage of the algorithm. This method of Logarithmic spiral prey is based on creating bubbles by encircling or through a '9' shaped path ^[27]. It can be modeled as follows:

3.1.   Searching prey

At this stage, the humpback whale does not know the position of the prey. The humpback whale can only change randomly to update its position. This means that humpback whales randomly select the position of an individual from the current population as prey to update its position. The humpback whales position is shown by the following Eqs (5)–(6):

where $\overrightarrow {{X_{rand}}}$ represents the random vector (a random whale). The coefficient vectors are represented by $\overrightarrow A$ and $\overrightarrow C$, and the calculation process is expressed as follows:

where the vector $\overrightarrow a$ is declining linearly from 2 to 0, and $\overrightarrow r$ is a random vector in the range of 0 and 1.

3.2.   Encircling prey

In the WOA, since it is impossible to know the location of the optimal solution in the search space, the algorithm assumes that the prey is the best candidate solution (the optimal whale location) or close to the optimal solution. After obtaining the best whale position, all whales update their positions towards the current optimal whale position. The calculation process shown as follows:

where current iteration is shown by $t$, ${X^*}$ represents the position of the optimal solution. if $\left| A \right| > 1$, the searching prey is represented by Eqs (5)–(6), otherwise the encircling prey is represented by Eqs (9)–(10), which expresses the shrinking mechanism.

3.3.   Logarithmic spiral updating position

At this stage, the distance between the individual whale and the optimal whale (prey) position is first calculated, and then the whale moves up the water surface in a spiral shape while spitting out many bubbles of varying sizes to prey. The whales are updated as follows:

where $\overrightarrow{{{D}^{'}}}=\left| \overrightarrow{{{X}^{*}}}\left( t \right)-\overrightarrow{X}\left( t \right) \right|$ shows the distance from whale to its prey, $l$ is a random number, which is the range of –1 and 1, the shape of the logarithmic spiral is defined as b, which is the constant number.

In the process of predation, whales have two ways of predation. Therefore, we assume that their probability of encircling mechanism is 50%, and the spiral model updates their positions with the same probability, and its model is as follows:

where $p$ is a random number, which is the range of 0 and 1. The pseudo-code of WOA can be displayed in Algorithm 1 below.

4.   An improved teaching learning-based whale optimization algorithm (TSWOA)

In the literature ^[36], the effectiveness of WOA in solving some real problems has been verified. Due to the role of leaders, the WOA has better search capability. These random leaders enable the whale algorithm to increase the diversity of the population in the early stages. However, the algorithm has problems such as slow convergence accuracy and difficulty in jumping out of local optimum. In addition, when researchers use WOA to solve complex real-world optimization problems, WOA needs some improvements to obtain better performance. Inspired by the characteristics of the teaching-learning-based optimization algorithm, we tried to improve the search ability of the WOA algorithm by utilizing the advantages of teaching-learning-based optimization algorithm (TLBO) ^[37].

It is called TWOA that the teaching phase of TLBO is added to the whale optimization algorithm. On the one hand, it enhances the exploitation capability of the algorithm and improves the quality of candidate solutions. On the other hand, from the biological background, whales are considered as highly intelligent animals with thinking, learning, judging, and communication. TWOA enables whales to have the ability of self-learning, which makes the algorithm more consistent with the biological mechanism. Meanwhile, the improved algorithm is more self-organizing and cooperative. We add the simplex method to the WOA to increase the diversity of the population and enrich the exploration capacity of the algorithm. The description of the new method is provided in the next subsections.

4.1.   Teaching learning–based optimization(TLBO)

TLBO is a metaheuristic algorithm whose idea comes from the traditional learning process to simulate the teaching of teachers and the learning of students in the class to achieve the purpose of optimization ^[37,47]. A class is a population in the search space, and the number of people in the class is the size of the population. The subject the student learns could interpreted as different design variables. The learning level represents the fitness of the optimization algorithm, and the teacher is the individual with the best fitness value of the population. There are two mechanisms for exchanging information in the classroom: collaboration between teachers and students and between students. These intra-class actions implemented in two different consecutive processes: the teacher phase is to simulate the influence of teachers on students, and the learner phase is to model the cooperative learning between students.

4.1.1.   Teacher phase

In the teacher phase, the students with the best fitness values are selected as teachers to improve the learning level of other students in the class. The average level of the class continues to improve, moving towards the optimal solution. The difference between the average level of students (population mean) and teachers is as follows:

where ${X_{{\rm{teacher}}}}$ is the individual with the best fitness value of the current population. ${X_{{\rm{mean}}}}$ is the mean value of the class, which is defined as ${{X}_{mean}}={\sum\limits_{i=1}^{n}{{{X}^{i}}}}/{n}$. The value of ${T_{\rm{f}}}$ can be either 1 or 2, which is again a heuristic step and decided randomly with equal probability as ${T_{\rm{f}}} = round(1 + rand(0, 1))$. In the teacher phase, the $i$-th learner in the class generates a new individual according to the following:

4.1.2.   Learner Phase

In the actual teaching process, the teacher's teaching is a heuristic process, and students may not be able to improve themselves effectively according to the teacher's teaching. Continuous learning from other students after class can improve their ability more effectively, which is the learning stage of the TLBO algorithm.

In the learner phase, randomly select two learners ${X_{\rm{i}}}$ and ${X_j}$ from the class, where $i \ne j$. The $i$-th learner in the class generates a new individual according to the following:

4.2.   Our approach (TSWOA)

4.2.1.   The combination of teacher phase and whale optimization algorithm

In this section, we mainly introduce the combination of teacher phase and whale optimization algorithm and we explain how to improve the quality of candidate solutions. To increase the exploitation, avoid falling into local optimum, and improve the quality of candidate solutions, we consider adding the teacher phase of TLBO to the algorithm to enable whale agents to have a capacity of self-learning. After each update of the whale agent, the fitness value of each whale agent was calculated. Then select the best whale agent and use formula 14 to guide other whale agents to move towards the current best position to improve the average performance of individuals in the population. The implementation steps of TWOA as follows:

4.2.2.   The Simplex Method (SM)

Although TWOA enhances the convergence precision of the WOA, it is easy to fall into the local optimum, so the Simplex method (SM) ^[38] is added to not only increase the diversity of population but also change the search direction of the algorithm and expand the search space. The simplex method obtains higher search accuracy by optimizing and has a good advantage in searching. The simplex method uses the optimization idea of reflection, expansion, and contraction of a convex figure (simplex) in multidimensional space to reach the minimum point ^[39]. Therefore, it is easy to help the algorithm to jump out of local optima. In this paper, after each update of the whale agent in the TWOA algorithm, the simplex method is run, where the specific addition position is shown in the pseudocode of Algorithm 3. The schematic diagram of the simplex method (Figure 2) is shown below:

The specific process of the simplex method is described as follows:

Step 1: Obtaining the objective function value of all points, and getting the best point as the ${X_{\rm{g}}}$, and then getting the second-best point as the ${X_{\rm{b}}}$, supposing that ${X_{\rm{s}}}$ is the one should be substituted. $f\mathit{(}{X_s}\mathit{)}$, $f\mathit{(}{X_b}\mathit{)}$ and $f\mathit{(}{X_{\rm{g}}}\mathit{)}$ show the objective function values.

Step 2: Obtaining the objective function value of the middle point ${X_c}$ between point ${X_g}$ and point ${X_b}$:

Step 3: Obtaining the reflection point ${X_{\rm{r}}}$ by using the following formula. $\alpha$ is the reflection coefficient, which is set as 1:

Step 4: If $f\left({{X_r}} \right) < f\mathit{(} {{X_g}} \mathit{)}$, getting the expansion point from formula 18:

where the expansion coefficient is $\gamma$, which usually set to 2. If ${\rm{f}}\mathit{(}{X_e}\mathit{)} < f\mathit{(}{X_g}\mathit{)}$, ${X_{\rm{s}}}$ will be replaced by ${X_e}$; otherwise, ${X_{\rm{s}}}$ will substitute ${X_{\rm{r}}}$.

Step 5: If $f\mathit{(}{X_{\rm{r}}}\mathit{)} < f\mathit{(}{X_s}\mathit{)}$, the compression point can be acquired by the formula (19):

where $\beta$ represents the cohesion coefficient, which is set to ${\rm{0}}{\rm{.5}}$. If ${\rm{f}}\mathit{(}{X_t}\mathit{)} < f\mathit{(}{X_s}\mathit{)}$, replace ${X_{\rm{s}}}$ by ${X_t}$; otherwise, ${X_t}$ substitute ${X_{\rm{s}}}$.

Step 6: If $f\mathit{(}{X_g}\mathit{)} < f\mathit{(}{X_r}\mathit{)} < f\mathit{(}{X_s}\mathit{)}$, shrink point is ${X_w}$, and the shrink coefficient is $\beta$.

If $f\mathit{(}{X_w}\mathit{)} < f\mathit{(}{X_s}\mathit{)}$, substitute ${X_s}$ by ${X_w}$; otherwise, substitute ${X_s}$ by ${X_r}$.

According to the simplex method above, this method diversifies the population, ensuring the algorithm can search efficiently, and reducing the algorithm fall into local optimum. The implementation steps of TSWOA are as follows.

The corresponding to the TSWOA flowchart as follow (see Figure 3):

5.   Using TSWOA to train MPL

In this part, we focus on ways to solve the problem. It covers the setting of the candidate solution dimension, the setting of the objective function and the setting of assessment to the experiment results.

5.1.   The setting of the dimensionality of the candidate solution

Before optimizing MLP, we must first obtain the structure of the MLP network. The number of neurons of the input layer and the output layer is decided by classified data sets, while the number of neurons of the hidden layer is decided by the Kolmogorov theorem ^[40]. The result is determined by Eq (21).

when TSWOA is adopted to obtain the connection weights and bias values, $D$ represents the dimension of the candidate solution. The result is determined by Eq (22):

where the number of neurons of the input layer, hidden layer, and output layer is expressed by Input, Hidden and Output. The number of bias of the hidden layer and the output layer is shown by $Hidde{n_{bias}}$ and $Outpu{t_{bias}}$.

5.2.   Setting of candidate solution form

In the study ^[41], we can get many encoding ways to optimize MPL. The mathematical models of the TSWOA algorithm are all represented by vectors, so we use vectors to represent. $X = \left\{ {{X_1}, \, {X_2}, \, {X_3}, \, \ldots, \left. {{X_n}} \right\}} \right.$ represents $N$ agents in the population. Each agent in population is denoted by ${X_i} = \left\{ {iw\, , \, hw\, , \, hb\, , \left. {ob\, } \right\}\, , \, i = 1\, , \, 2\, , \, \ldots, \, n\, , } \right.$ the weights of the input layer and the hidden layer are denoted by $iw$ and $hw$, the biases of the hidden layer and the output layer are shown by $hb$ and $ob$.

5.3.   The setting of objective function

In optimizing MLP, it is crucial to define appropriate objective functions. Generally, after agents are defined by vectors, the objective function of the neural network needs to be used to define the fitness function of the TSWOA algorithm. Hence, the fitness function is considered as the difference between the theoretical output and the actual output in the neural network, which is obtained by using Eq (23):

where the total number of output is shown by $m$, the desired output and the actual output of $i$-th input are exhibited by $d_i^k$ and $O_i^k$.

Apparently, training neural networks requires data sets, and there are many training data sets. So, the fitness function is the mean value of MSE, which is obtained by using formula 24:

where the total number of training data sets is shown by $s$, other parameters are the same as the formula above. Therefore, the objective function of optimizing MLP is defined:

5.4.   Setting of assessment

Because classified data is used to optimize MLP, classification problems often involve learning existing data to forecast undiscovered data. In classification problems, accuracy is considered to assessing the classification capacity of the algorithm, which is defined as follows:

where $\overline N$ shows the correct number for the classification, the total number of datasets is denoted by $N$.

6.   Experiment and result analysis

In this section to test the validity of the algorithm for optimizing MPL, fifteen data sets from the UCI machine learning knowledge were obtained. There are 15 data sets displaying in Table 1. To better test the performance of TSWOA, the other six state-of-the-art algorithms are adopted: grasshopper optimization algorithm (GOA) ^[19], glowworm swarm optimization (GSO) ^[42], social spider optimization algorithm(SSO) ^[43], flower pollination algorithm(FPA) ^[19], genetic algorithm (GA) ^[11], and whale optimization algorithm(WOA) ^[44].

6.1.   Experimental setup

All used codes in this study have been implemented in the same manner using MATLAB R2017 (a) and run on a PC with the Windows 10 64-bit and Intel Core (TM) i5-4590 processor, 3.30 ghz and 4 GB RAM. For all data sets above, data for training and testing respectively accounted for 66% and 34% of the total data ^[45]. To obtain great accuracy of experimental results, each algorithm runs 20 times. The number of iterations is 500 in independent experiments. The population number is 30 and the main parameter settings for all algorithms are in Table 2.

6.2.   Experimental results and analysis of training MLP

The 15 data sets were obtained from the UCI machine learning knowledge, namely, Blood, Scale, Survival, Liver, Seeds, Wine, Iris, Statlog, Cancer, Diabetes, Gene, Parkinson, Splice, WDBC, and Zoo. The experimental results are displayed in Tables 3–17, in which the "Best" is the best value, the "Worst" is worst value, the "Mean" is the mean value and the "Std" is the standard deviation. Accuracy is the best accuracy of 20 independent runs, and rank is the algorithm ranked according to accuracy. Due to the randomness of the swarm intelligence algorithm in the process of operation, the statistical test is important ^[46]. In order to obtain the difference between the improved algorithm and contrastive algorithms, Wilcoxson's rank-sum test is also added in the experiment to obtain whether there is an obvious difference between the two groups of data. When the value is less than 0.05, the difference between the two groups of data is obvious. The $p$ values are revealed in Table 18.

6.2.1.   Blood classification problem

The blood classification problem is the most commonly used data set in the classification problem. The experimental results of the classification problem of Blood dataset trained by intelligence optimization algorithm are shown in Table 3, and the convergence and the anova graphs of algorithms are displayed in Figures 4 and 5, severally. It was found from Table 3 that the mean value and variance of MSE obtained by the TSWOA algorithm are better than other comparison algorithms, indicating that the optimizing result of the TSWOA algorithm is the best. Experimental results show that the accuracy of the algorithm is 80.39, higher than that of other comparison algorithms, ranking first. As can be found from the convergence curve in Figure 4, the convergence speed of the TSWOA algorithm is faster than that of other algorithms and the accuracy value is better than that of other algorithms. According to Figure 5, compared with other comparison algorithms, TSWOA has a better variance value. According to the $p$-value test shown in Table 18, the results of TSWOA is obviously different from other comparison algorithms.

6.2.2.   Scale classification problem

The experimental results of 20 independent runs of the algorithm are displayed in Table 4, and the convergence and anova graphs of all algorithms are displayed in Figures 6 and 7, severally. As displayed in Table 4, the mean value of MSE obtained by TSWOA algorithm is the smallest in the data set. But, the accuracy of the TSWOA algorithm and the FPA algorithm in optimizing MLP to forecast Scale data set is 89.67, ranking first, and their classification accuracy is far higher than that of other comparison algorithms. As seen in Figure 6, the TSWOA algorithm has faster convergence speed and higher convergence accuracy than other comparison algorithms. Figure 7 exhibits that TSWOA has a small and stable anova. Table 18 also shows the superiority of TSWOA, which is obviously different from all comparison algorithms in Scale dataset.

6.2.3.   Survival classification problem

Table 5 displays the experimental results of the seven algorithms that optimized MLP to forecast the Survival dataset. The convergence and variance graphs of all algorithms are displayed in Figures 8 and 9, severally. As displayed in Table 5, the mean value of MSE obtained by the TSWOA algorithm is the smallest in the comparison algorithms. But, the accuracy of the TSWOA algorithm optimizing MLP in forecasting the Survival dataset is lower than that of GOA, GSO, SSO, and GA, which ranks third. Among them, the classification accuracy of GOA, GSO and SSO algorithms is 81.73, ranking first. The convergence curve in Figure 8 shows that the TSWOA algorithm has a faster convergence speed and a better convergence accuracy value than other algorithms. In Figure 9, compared with other algorithms, TSWOA has a better mean value. Table 18 displays that the TSWOA algorithm has obvious advantages.

6.2.4.   Liver classification problem

The experimental results of the Liver classification problem are shown in Table 6, and Figures 10 and 11 display the convergence and variance graphs of all algorithms, severally. As displayed from Table 6, the mean value of MSE obtained by the TSWOA algorithm is slightly smaller than the mean value of MSE obtained by GOA and WOA. In the Liver dataset, the accuracy of the TSWOA algorithm was higher than that of other comparison algorithms, ranking first. It can be distinctly displayed from Figure 10 that at the beginning of the iteration, the TSWOA algorithm not only converges faster but also has a better mean value of MSE than other comparison algorithms. As shown in Table 18, the p value of the TSWOA algorithm and other algorithms are all less than 0.05, indicating the excellent performance of the modified algorithm in processing the Liver dataset.

6.2.5.   Seeds classification problem

The MSE results obtained by all algorithms listed in Table 7 display that there is little difference between the mean values obtained by the TSWOA algorithm and the FPA algorithm, and the performance of the two algorithms is much better than other algorithms. Meanwhile, the accuracy of both the TSWOA algorithm and the FPA algorithm is 95.77, which is far higher than the accuracy of other comparison algorithms. Figure 12 exhibits that the convergence rate of the TSWOA algorithm is much faster than that of GOA, GSO, SSO, GA, and WOA, but slightly worse than that of FPA. Before 250 iterations, the convergence speed and convergence accuracy of the TSWOA algorithm was much higher than that of other comparison algorithms, but in the later iteration period, the convergence accuracy of the FPA algorithm was slightly higher than that of TSWOA algorithm. Wilcoxson's rank-sum test in Table 18 shows that the value of the TSWOA algorithm and the FPA algorithm are greater than 0.05, but the comparison results of other comparison algorithms are less than 0.05.

6.2.6.   Wine classification problem

The results in Table 8 display that the TSWOA algorithm obtains better results than the WOA algorithm and the other five algorithms. Table 8 shows that, as far as MSE is concerned, the mean value of TSWOA algorithm is the smallest and the classification accuracy of the algorithm is 96.72, which is much higher than that of other comparison algorithms, indicating that TSWOA algorithm can achieve better classification accuracy than other comparison algorithms. Figure 14 exhibits the convergence curve of the average MSE of all the seven algorithms, which distinctly indicates that GOA, GA, and WOA quickly fall into local optima and their convergence speed has stopped when the number of iterations is 300. Meanwhile, about 500 iterations of FPA and TSWOA reached the corresponding minimum value, but the MSE of TSWOA was smaller than that of FPA. According to the test results in Table 18, the results of the TSWOA algorithm are significantly different from that of GOA, GSO, SSO, GA and WOA algorithms.

6.2.7.   Iris classification problem

The experimental results of the Iris classification problem are exhibited in Table 9, and Figures 16 and 17 display the convergence and variance graphs of all algorithms, severally. The mean value of MSE obtained by the algorithm shown in Table 9 exhibits that the value obtained by FPA is superior to that of other comparison algorithms, while the value obtained by TSWOA ranks the second. However, the accuracy obtained by TSWOA and GA is much higher than other algorithms. Figure 16 shows that before the 240 iterations, the convergence speed and convergence accuracy of the TSWOA algorithm is much higher than that of other comparison algorithms. But, after the 240 iterations, the convergence accuracy of FPA is greater than that of TSWOA. Meanwhile, GOA, GA, SSO and WOA algorithms quickly fall into local optimization. As displayed in Figure 16, TSWOA has a better variance value than other algorithms. According to the p-value test in Table 18, the results of TSWOA are obviously differences from those of GOA, GSO, SSO, GA, and WOA.

6.2.8.   Statlog classification problem

The results in Table 10 exhibit that, in terms of MSE, the mean value and variance of the TSWOA algorithm are the smallest compared with other comparison algorithms. At the same time, the accuracy in Table 10 indicates that the TSWOA algorithm can achieve a better classification effect than other comparison algorithms. Figure 18 displays the convergence curves of MSE obtained by all the seven algorithms, which clearly shows that GSO, GA, SSO, and WOA quickly fall into local optima and their convergence speed has stopped when the number of iterations is 200. However, the TSWOA algorithm has been in a declining state in the process of 500 iterations. Figure 19 exhibits that the TSWOA algorithm has a relatively stable variance. Table 18 also displays the superiority of TSWOA, which is obviously different from all algorithms.

6.2.9.   Cancer classification problem

Figure 20 exhibits the convergence of the seven algorithms. As displayed from Figure 20, compared with other comparison algorithms, TSWOA obtained the minimum mean value. FPA is approximately the same mean value obtained by WOA, while the convergence accuracy and speed of the TSWOA algorithm are better than other algorithms. At the same time, it can also be observed that during the optimization process, GA and GOA obtained the same minimum mean, while GSO performed the worst, stagnating in the early stage. Table 11 shows the comparison of the mean values of different algorithms. It can be clearly found from Table 11 that after 500 iterations, the mean value obtained by TSWOA is the smallest. WOA and FPA obtained the same classification effect and ranked first, while TSWOA, GA, and SSO obtained the same classification effect and ranked second. The results above show that TSWOA provides very competitive results in the classification effect and the p-value in Table 18 further displays that the TSWOA algorithm is significantly superior to all comparison algorithms in the classification effect of the Cancer dataset.

6.2.10.   Diabetes classification problem

The experimental results of the diabetes classification problem are displayed in Table 12, and Figures 22 and 23 exhibit the convergence and variance of all algorithms, severally. As exhibited from Table 12, as far as MSE is a concern, TSWOA provides the best mean value and the standard deviation of MSE, the result is better than other comparison algorithms, and the accuracy of classification of Diabetes data set is better than that of other comparison algorithms. Figure 22 shows that TSWOA not only has a fast convergence rate but also has a better mean value than other comparison algorithms. In addition, In Figure 23, TSWOA has the smallest variance value. Table 18 also displays the superiority of TSWOA, which is significantly different from all comparison algorithms.

6.2.11.   Gene classification problem

The results in Table 13 display that the mean valve of MSE obtained by the TSWOA algorithm is better than that of the WOA algorithm and the other five algorithms. The accuracy of the modified algorithm is 33.33, which is much higher than that of other algorithms and ranks first, indicating that the training of the Gene dataset with the TSWOA algorithm can achieve better classification accuracy than other comparison algorithms. Figure 24 exhibits the convergence graphs of all the seven algorithms, which distinctly exhibit that the three algorithms, GSO, GOA, and GA, quickly fall into local optima and their convergence speed has stopped in the early stage of the iteration. However, during the optimization process, TSWOA did not stop still and continued to search for the appropriate minimum. Figure 25 also shows that TSWOA has superior performance. Table 18 displays that TSWOA is obviously different from the other six algorithms. In this case, all values are less than 0.05.

6.2.12.   Parkinson classification problem

The experimental results of Parkinson's classification problem are shown in Table 14, and the convergence and variance of all algorithms are exhibited in Figures 26 and 27. It displayed from Table 14 that the mean value of MSE obtained by the proposed algorithm is the minimum, indicating that the training effect of the improved algorithm is the best. In the test data, the accuracy of the modified algorithm is 74.24, higher than that of other algorithms, ranking first. In Figure 26, the convergence speed of our algorithm is faster than that of other algorithms. It displayed in Figure 27 that TSWOA has a better variance value than other algorithms. Two abnormalities appeared in the experiment, whose average value was less than that of other algorithms. The p-value test results in Table 18 further indicate that the results of TSWOA are obviously superior to all comparison algorithms.

6.2.13.   Splice classification problem

Table 15 displays the experimental results of the splice classification problem, the convergence and variance diagrams of all algorithms are exhibited in Figures 28 and 29, severally. It displayed in Table 15 that the mean value of MSE obtained by the modified algorithm is the minimum. The mean values obtained by GOA and GSO are almost the same. In the Splice dataset, the accuracy of the proposed algorithm is 80, which is higher than the accuracy of other algorithms, so it ranks first. In Figure 28, our algorithm has a great advantage over other algorithms in terms of convergence speed and convergence accuracy. Meanwhile, GSO, GOA, and GA soon fell into local optima. In Figure 29, compared with other algorithms, TSWOA has a better variance value. Experimental results found no singularities, and the mean value of the TSWOA algorithm was lower than that of the other algorithms. In Table 18, $p$-value test results display that the TSWOA algorithm is superior to all comparison algorithms.

6.2.14.   WDBC classification problem

The results in Table 16 show that, as far as MSE is concerned, the mean value and variance of the TSWOA algorithm are the smallest, indicating that the TSWOA algorithm can achieve better classification effect than other comparison algorithms. In the test data, the accuracy of the modified algorithm is 95.15, which is higher than the accuracy of other algorithms, so it ranks first. Figure 30 exhibits that GSO, GA, GOA, and WOA fall into local optimality early in the optimization process, resulting in premature convergence of the algorithm. However, in the optimization process, TSWOA did not stop and continued to search for the corresponding optimal value. All the results exhibit that TSWOA is superior to other algorithms, with the fastest convergence speed and the smallest value. As can be displayed in Figure 31, compared with other algorithms, TSWOA has a better variance value. It can be concluded in Table 18 that TSWOA is obviously different from other comparison algorithms.

6.2.15.   Zoo classification problem

The MSE results obtained by the algorithm di splayed in Table 17 exhibit that there is little difference between the mean values obtained by FPA and TSWOA, and the performance of these two algorithms is much better than that of other algorithms. However, the classification accuracy of the modified algorithm is slightly lower than that of WOA in test data. In the Zoo dataset, the accuracy of the modified algorithm is 64.71, which ranks second. The convergence of the seven algorithms is displayed in Figure 32. As can be shown from Figure 32, in the later iteration, TSWOA, WOA and FPA obtained convergence accuracy with little difference. However, GSO and GA tend to fall into local optima in the early stage. As can be seen from Figure 33, compared with other algorithms, TSWOA has a good mean value in training data. The results above exhibit that TSWOA provides very competitive results. It can be concluded from Table 18 that, except for FPA, the values obtained by TSWOA are significantly different from the other five algorithms.

6.3.   Result analysis

As can be seen from Tables 3–17, except Survival, Cancer, and Zoo, the accuracy of the modified algorithm is the greatest, indicating that the classification failure rate of TSWOA is small in optimizing MLP. In terms of Survival, Cancer, and Zoo, the accuracy of TSWOA is also in the top three. Moreover, the rest are ranked first. From Figures 4–33, it can be seen that the improved algorithm has faster convergence speed, higher accuracy, and stronger stability for optimizing MLP. The reasons are as follows:

For any optimization algorithm, it is important to obtain the balance between its exploration and exploitation. Exploration makes the algorithm to have feasible solutions and exploitation provides search intensity for any optimization problem. As displayed in section 3. It is vital to have the arguments $\alpha$ in the position update of WOA. By changing the value of the arguments $\alpha$, a better balance can be realized between the global search and local search. However, in the WOA, $\alpha$ is declining linearly from 2 to 0 with consistently of the iterations. At the beginning of the iteration, the arguments $\alpha$ can be a max value that is beneficial to global search, but the algorithm has a poor search efficiency. At the end of the iteration, a small value is acquired, which is beneficial to the convergence of the algorithm, but the algorithm is prone to a local optimum. Based on the problems above, the teaching phase of TLBO is add to the basic whale optimization algorithm and selects the best agent to guide other agents to move towards the current optimal position. On the one hand, we heighten the exploitation capacity of the algorithm and improve the quality of candidate solutions. On the other hand, it evades the stagnation state of the algorithm and decreases the risk of the algorithm running into local optimum. In order to strengthen the exploration capacity of the algorithm, the simplex method is added to the algorithm, which not only increases the variety of the population but also enlarges the search scope of the solution space, guiding the algorithm to find the most promising area of the solution space. Therefore, this modified algorithm can be applied as a new MLP optimizing method.

7.   Conclusions and future directions

In this paper, an improved teaching learning-based whale optimization algorithm was presented to optimize MLP. The modified TSWOA algorithm utilizes the teacher phase of TLBO to heighten the exploration and increase the overall quality of candidate solutions. The simplex method is added to TWOA to enlarge the diversification of search agents, which ensures that the algorithm can availably explore the search location, and make the algorithm quickly converge to the global optimal. This method was adopted to optimize connection weights and biases of multi-layer perceptron (MLP) neural network for the classification problems, compared with several recognized meta-heuristic algorithms. The experimental results display that the modified TSWOA is quite effective in optimizing MLP. Future research may apply TSWOA to optimize convolutional neural networks for image processing. Another opportunity is to further heighten the performance of TSWOA to get a better balance between exploration and exploitation and apply it to solve the real problems.

Acknowledgments

This work is supported by National Science Foundation of China under Grants No. 61563008, and Project of Guangxi Natural Science Foundation under Grant No. 2018GXNSFAA138146.

Conflict of interests

The authors declare no conflict of interest.