Improving diversification by a hybrid bat-Nelder-Mead algorithm and DDE for rapid convergence to solve global optimization

1.   Introduction

The discovery of several real-world applications is crucially related to delay differential equations and algorithms. Different algorithms are designed to solve global optimization problems. Global optimization problems are inherently intractable. Conventional algorithms (deterministic algorithms) are not efficient to solve global problems. One of the main reasons for this is that deterministic algorithms seek the exact global optimum among all feasible regions, which is a time-consuming procedure for problems of moderate to high dimension. Ideal surrogate for deterministic algorithms in solving these problems are algorithms with a stochastic element, called heuristic algorithms, as the randomness in the global search guides the search toward regions where a better solution is likely to be found. Metaheuristic algorithms are a type of heuristic applicable for a wide range of problems. In the past few decades several nature-inspired metaheuristics have been developed. One class of them is based on the social interactions of animal communities, called swarm intelligence (SI) algorithms, such as the genetic algorithm ^[1], particle swarm optimization (PSO) algorithm ^[2], ant colony optimization (ACO) ^[3], artificial fish swarm algorithm (AFSA) ^[4], and bat algorithm (BA) ^[5]. Recently, several other metaheuristic algorithms were proposed such as the liver cancer algorithm (LCA) ^[6], fata morgana algorithm (FATA) ^[7], snow geese algorithm (SGA) ^[8], and more.

Over time, diligent research has shown some defects that affect the performance of standard metaheuristics in solving certain problems. Accordingly, numerous impressive algorithms were reported that combine metaheuristics with different techniques or other algorithms to improve new versions that overcome the deficiencies in the component algorithms. In ^[9], the author combined the firefly algorithm (FA) with extremal optimization to tackle the defects of FA in slow convergence and falling into local minimum. In ^[10], the author built a framework for conducting an effective analysis for recurrent spontaneous abortion. This framework was presented by a combination between the joint self-adaptive sime mould algorithm (JASMA) with the common kernel learning support vector machine with maximum-margin hyperplane theory. BA was not excluded from this cutting-edge phenomenon. The BA has some advantages over other algorithms that make it promising and interesting such as its simplicity in structure, speed in processing, and ease in hybridization with other algorithms. Furthermore, the BA efficiently succeeded in dealing with several optimization problems in different fields such as power systems, economic load dispatch problems, image processing, and medical applications, see ^[11] for a detailed description. However, the exploration in the BA is weak as the BA focuses in the exploitation rather than exploration which makes it fall into local optimum. For that, many works proposed variants of the BA to improve its performance by introducing different techniques. For instance, the researcher in ^[12] incorporated a chaotic sequence and chaotic Levy flight schemes to enrich the searching behavior and improve the local and global search capabilities. In ^[13], the author introduced, first, the iterative local search in the local search of the BA to avoid falling into local minimum and, second, a stochastic inertia weight in the velocity updating formula of the BA. The researcher ^[14] incorporated quantum evolution and an annealing strategy into the BA to achieve a balance between intensification and diversification. Other works tend to combine the BA with other algorithms in which the combination is referred to as hybrid metaheuristics. This hybridization accentuates the mettle of its components and creates an algorithm capable of solving many hard optimization problems. For instance, ^[15] introduced a step in the harmony search (HS) algorithm in the BA and added an HS attribute as an operator. Author in ^[16] introduced a hybrid strategy that combines the genetic algorithm (GA) with BA to improve the efficiency of the global search of the standard BA. In their work, the mean magnitude of relative error was applied as an objective function. Other variants of the BA were proposed by combining the benefits of the BA with the local search Nelder-Mead algorithm (NM). In ^[17], the author started by following the BA routine to generate a new set of solutions. If any inferior solution was obtained, NM was performed by taking the inferior solution as the starting point of the simplex to refine it. While, in ^[18], the researcher took the best solution obtained after performing the BA routine as the starting point of the simplex in the NM algorithm. In ^[19], the NM method was used as a local search instead of the random walk method that is applied in the standard BA to refine the best solution found so far, which helped to accelerate the search process. In ^[20], the author replaced the random walk that is used as a local search in the standard BA by the pattern search method to increase the intensification ability. Furthermore, the NM method was employed in the final stage of the algorithm to improve the best found solution.

The motivation of this work is to tackle the main drawback of the BA, which is the weakness in exploration, by making changes in its structure, specifically in the updating process, using the NM algorithm rather than the technique used in ^[17,18,19], which is refining the solution found by the standard BA utilizing the NM method. For that, a novel hybridization is carried out utilizing the BA and NMA at which the global search ability of the standard BA is enhanced in the proposed algorithm (HBNMA) in premier iterations by the intervention of the NMA in the updating process of the solutions. This helped at first to distract solutions from heading only toward best solution found so far to avoid falling into local minimum. Second, it helped to penetrate deeply in a region where further potential best solutions may be found using sequential expansion steps that are defined in the NM algorithm. This can be seen in the Experiments and results section. By the lapse of iterations, the exploitation arises by the usual local search mechanism of the classic BA. This balance between exploration and exploitation enables HBNMA to solve complex global optimization problems. The contributions of this paper are as follows:

● The fast hybrid bat-NM algorithm is proposed with a new contribution of the NM in a hybridization. The NM in this work is used to tackle the main drawback of the BA by improving the updating process of the BA. The proposed algorithm is able to solve complex and high-dimensional global problems.

● Breaking the tradition of following a uniform route for all particles, in the proposed algorithm, particles determine their next step by choosing whether to follow the proposed hybrid updating process or the classic one of the standard BA according to which one is able to achieve better results.

● Precise analysis of the working mechanism of the proposed algorithm and thoughtful experiments are presented with statistical measurements to perfectly evaluate the merit of the HBNMA.

The rest of this paper is organized as follows: Section 2 gives an overview on the classic NM and classic BA. Furthermore Section 2 describes the proposed algorithm. In Section 3, experiments and numerical results are illustrated. The time complexity is described in Section 4. In Section 5, the study's conclusion is presented. Finally, future work is described in Section 6.

2.   Materials and methods

2.1.   Classic Nelder-Mead algorithm (NM)

In this section, the classic NMA is presented as stated in the original version ^[21]. The main thrust of the NM is to solve a multidimensional, unconstrained optimization with no need for derivatives.

where the objective function $f:R^{d}\rightarrow R$ and $x\in R^{d}$.

This algorithm constructs a design framework called simplex with $d+ 1$ vertices for a d-dimensional problem, in which the worst vertex is replaced by a better one at each iteration. The surrogate vertex is obtained utilizing one or more procedures including reflection, expansion, contraction, or reduction.

The NM starts by initializing the points $x_{i}$ (the vertices of the simplex). Next, the fitness value (objective function) $f$ is evaluated for each point $x_{i}$. The points are reordered according to their objective function value to determine the best $x_{b}$, the second worst $x_{w2}$, and the worst $x_{w}$, which satisfy:

After that, the centroid $x_{c}$ of all points except $x_{w}$ is calculated using:

The NMA implements different procedures to decide the most effective step in a direction expressed utilizing a line segment joining $x_{c}$ with $x_{w}$. The procedures are described in the following in which the values of the hyperparameters $\mu_{r}, \mu_{e}, \mu_{oc}, \mu_{ic}$, and $\mu_{re}$ that are used in these procedures are assigned to the same values as in ^[21].

$(1) ~~ \mathit{\boldsymbol{Reflection}}$:

This step indicates a reflection of the point $x_{w}$ using the point $x_{c}$. The reflection point $x_{r}$ is calculated utilizing the following equation:

$(2)~~ \mathit{\boldsymbol{Expansion}}$:

This step expands the reflection step as long as an improvement is achieved. The expansion point $x_{e}$ is calculated utilizing:

$(3)~~ \mathit{\boldsymbol{Contraction}}$:

This step shrinks the size of the simplex utilizing either an outside contraction $x_{oc}$ or inside contraction $x_{ic}$. The points $x_{oc}$ and $x_{ic}$ are calculated as follows:

$(4)~~ \mathit{\boldsymbol{Reduction}}$:

This is the last resort after the failure of all previous attempts. In this step, all points $x_{i}$ are updated toward the best point $x_{b}$ using:

2.2.   Classic bat algorithm (BA)

The bat algorithm emulates the echolocation trait and hunting behavior of microbats. ^[5] designed the classic BA from the following idealized rules:

(1) Different bats utilize echolocation to sense the distance between them and the target with a magical ability to distinguish between food and background barriers.

(2) Bats fly randomly with velocity $v_{i}$ at position $x_{i}$ with a fixed frequency $f_{m}$ and a varied wavelength $\lambda_{i}$ and loudness $A_{i}$ to seek out prey. They can adjust automatically their wavelength (or frequency) according to the proximity of their target.

(3) Loudness is assumed to vary from a large value $A_{0}$ to a minimum value $A_{min}$.

According to these rules, the BA is designed as follows:

(1) Initialize randomly a population of $n$ solutions $x_{i}\in R^{d}$.

(2) Update the frequency $f_{i}$ and the velocity $v_{i}$ using:

where $f_{min}$ and $f_{max}$ are prespecified minimum and maximum values for $f$, $rand$ is a random number, $t$ is the current iteration, and $x_{*}$ is the current global solution. Then a new solution $x_{i}$ is generated as follows:

(3) If $rand < r_{i}$, evaluate a new local solution around the current global one using:

where $\epsilon \in [-1, 1]$, and $A^{t}$ is the average loudness of all solutions at iteration $t$.

(4) Update the best individual solutions and both the loudness and rate of pulse emissions using the following formulae if an improvement is achieved:

where $\alpha \in (0, 1)$, $\gamma >0$, $A^{t}_{i}\rightarrow 0$, and $r^{t}_{i}\rightarrow r^{0}_{i}$, as $t \rightarrow \infty$.

(5) Select the current global solution from the best individual solutions.

2.3.   Proposed HBNMA

A novel hybridization strategy between the BA and NMA is introduced in this section called the HBNMA. The standard BA guides particles toward the best solution whereas the standard NM guides the worst particle (vertex) toward the centroid, or the center of gravity, of the other particles. The HBNMA is based on the concept that a solution at the beginning of the iterations should experience a different direction besides the direction toward the best solution in order to avoid falling into local minimum. Accordingly, the HBNMA developed a new updating formula for the velocity of a particle composed of the usual direction of the BA and the direction of the NM in order to enhance the global search ability of the proposed HBNMA.

The proposed algorithm starts by initializing $n$ solutions $x_{i}$ randomly using:

where $[x_{min}, x_{max}]$ is the range of the search space, $i = 1, 2, ..., n$, and $rand \in (0, 1)$ is a random number. Further, the velocity, frequency, loudness, and pulse emission rate are initialized. Utilizing the solution's initial information, the HBNMA evaluates the fitness value. The solution with the best fitness is stored as a current global solution $x_{*}$.

Each particle has two choices from which its step can be determined. The first is a combination between the classic BA step $v_{bat}$ (2.9) and the reflection step $v_{r}$ of the NM:

where $x_{c}$ is defined in (2.2). Figure 1 shows this combination geometrically in green-colored lines or more detailed in blue-colored lines that generate the step from $v^{t}$ to $v^{t+1}$. Hence the improved updating formula of each particle's velocity $v_{i}$ becomes:

If progress is achieved in the fitness value for $x_{i}$, that is, $f(x_{i}^{t+1}) < f(x_{i}^{t})$, sequential expansion steps are carried out as long as progress is made to exploit the area deeply. The expansion steps are performed by replacing $v_{r}$ in (2.16) by $v_{e}$ which is calculated as:

with $\mu = 2*\mu_{pr}$, where $\mu_{pr}$ is the value of $\mu$ in the previous expansion step. The expansion steps are of the form:

The term $v_{bat}$ solely guides particles toward the current global solution as in the classic BA, as shown in Figure 1 with red-colored lines, which may make particles fall easily in a local optimum. Combining the terms $v_{r}$ or $v_{e}$ with $v_{bat}$ in (2.16–2.18), shown graphically in Figure 1, distracts the particle's concentration from the exclusive dependence on the direction toward the best one, especially at the beginning of the iterations. This helps to target new areas in the search space which may be promising. Accordingly, it allows one to effectively explore the region. This choice of the particle's step is accepted if progress in the fitness value is achieved. If not, the second choice is to keep the same updating process as in the standard BA (2.9). The proposed mechanism takes place at the beginning of the iterations. While in the later iterations, as is usually done in the standard BA, if $rand < r_{i}$, solutions are updated locally around the current global solution using (2.11). This local intensification occurs in the area where the global optimal is expected.

The full description of the HBNMA is shown in the following algorithm and flowchart (Figure 2):

3.   Experiments and results

In this section, we validate the efficiency of the proposed HBNMA and its counterparts using classical and complex benchmark test functions along with the Wilcoxon signed rank test. Furthermore, the working mechanism of the HBNMA is analyzed numerically.

3.1.   Benchmark test functions

The performance of the HBNMA is examined utilizing two sets of test functions which provide a good representation of real-world optimization problems and show the merit of the proposed algorithm. The first set is the classical test functions whose definitions are found in ^[22]. The selected functions vary in their characteristics. Some are unimodal to test the intensification ability of the proposed algorithm while the others are multimodal functions to test the ability to jump out of the local optimum. Moreover, the selected test functions vary in separability that measures the difficulty of the problem. The dimensions of the selected functions, the range used, their optimum values, and their characteristics are described in the table found in the Supplementary Materials. The second set is the IEEE CEC2014 special session and competition on a single-objective real-parameter, numerical optimization benchmarking suit ^[23]. This set of complex test functions contains 3 rotated unimodal functions F1–F3, 13 rotated and shifted multimodal functions F4–F16, 6 hybrid functions F17–F23, and 8 composition functions F24–F30 that validate the performance of the proposed algorithm.

3.2.   Statistical analysis

The significance of the proposed HBNMA is measured using some evaluation metrics and the Wilcoxon signed-rank test on the results of the experiments in Section 3.3.

3.2.1.   Evaluation metrics

The performance of the proposed HBNMA in optimizing an objective function is evaluated using three metrics which are the best value (BV), the mean value (MV), and the standard deviation (SD) over several runs.

3.2.2.   Wilcoxon signed-rank test

This test is a non-parametric statistical test that aims to detect whether there is a significant difference between two sample means or not.

In this research, the Wilcoxon signed-rank test is applied at a significance level of $\alpha < 0.05$. The sum of the ranks for the functions in which the HBNMA outperformed its counterpart is denoted by $R^{+}$. While the sum of the ranks for the opposite is denoted by $R^{-}$. The two criteria $R^{+}$ and $R^{-}$ are calculated as described in ^[24] in which the average ranks are used in dealing with ties. The Wilcoxon test produces a test statistic value (Z) which is converted into a probability (p-value). A low p-value ($p < 0.05$) reflects a significant difference between the HBNMA and its counterpart. IBM SPSS software is used in this work to calculate the p-value.

3.3.   Experiments and discussions

In this section, three experiments are performed to show the merit of the proposed HBNMA on different dimensional problems. The first experiment is a comparison against the standard BA using problems with $d = 5$ up to $d = 1000$ for each to show the positive impact of the hybridization of the BA with the NMA in the performance of the proposed algorithm. Furthermore, Experiment 1 provides an analysis of the positive impact of the hybridization numerically. The second experiment is a comparison against different variants of the BA that is described in Section 3.3.2 using standard benchmark test functions with different properties to perfectly validate the proposed algorithm. The third experiment is against state-of-the-art heuristics utilizing CEC2014. The results obtained from these experiments are evaluated utilizing three evaluation metrics and the Wilcoxon signed-rank test as described in Section 3.2. The code of the HNMBA was written and run in MATLAB 2013, on Windows 10 with an Intel ® Core i5-1035G CPU processor and 8.00 GB RAM.

3.3.1.   Experiment 1 (Analysis of the HBNMA working mechanism with a comparison utilizing high-dimensional problems)

This experiment is divided into two parts. The first part is a comparison between the proposed variant of the BA which is the HBNMA and the standard BA on low- and high-dimensional problems to show the effect of the hybridization of the BA with the NM in the performance of the HBNMA. The second part is an analysis of the algorithm's working mechanism which gives an explanation of the good results obtained for the HBNMA.

Tables 2 and 3 show the results of the comparison utilizing several test functions of dimensions $d = 5, 10, 100$ and $1000$ for each. The experimental parameter values are the same in this comparision for both the BA and HBNMA to ensure fairness and they are shown in Table 1. The results are evaluated using the error between the global optimal and the obtained solution, the mean number of function evaluations $NFE$ of 40 independent runs needed to obtain the accuracy indicated, as well as the average time taken to perform one run. Each algorithm terminates if either the error equals zero or the maximum number of function evaluations is reached ($MaxFE = 2*10^4$). The results showed the ability of the proposed HBNMA in convergence to the global optimal with zero error and a very low number of function evaluations in most functions with low or high dimension. In contrast, the BA failed to obtain good results in most indicated functions, even though it took the prespecified maximum number of function evaluations, at which the results got worse with an increasing function dimension as illustrated in Tables 2 and 3 and clarified in the example shown in Figure 3 using the Schwefel function. The figure demonstrated that the error value was getting worse in the BA with the increase in dimension, up to 90 in the left-hand figure and from 100 up to 800 in the right-hand figure, because of its weakness to handle the high dimensionality of the problems. Meanwhile, the performance of the HBNMA with the increase in dimension was stable with an error equal to zero. This comparison indicates that the hybridization in the HBNMA at first enhances the exploration ability in the proposed algorithm than that in the standard BA. This enhancement enables the HBNMA to search deeply for the global optimum and jump out from any local optimum no matter the dimension of the test function, as illustrated in the results of the error values of the HBNMA and BA. Second, the hybridization decreases the time consumed to reach the global optimum in the HBNMA from that in the BA. This proves the merit of this work.

Further in this experiment, the behavior of the particles in the HBNMA is demonstrated using four metrics to give a clarification to the superiority of the algorithm. Table 4 illustrates the percentage of the mean number of particles that follow the improved updating formulae (2.16) and (2.18) ($M_{im}$) in several iterations in one random run and those that follow the standard updating formula of the BA (2.9) ($M_{ba}$). Also, the success in any iteration ($t+1$) to achieve an improvement in the global solution ($f(x_{*}^{t+1}) < f(x_{*}^{t})$) is studied whether referring to a particle that follows the improved formulae or the standard formula. Then the percentages of the successes are calculated and denoted in Table 4 as ($Suc_{im}$) or ($Suc_{ba}$), respectively. The results indicated that the majority, around two-thirds, of the particles follow the standard updating process of the BA. The reason behind this is that the particles follow the improved updating process if progress in their previous fitness value is achieved which is not continuously done. However, the high percentage of success in improving the global solution is due to particles that follow the improved updating process in all multimodal and some unimodal functions. The high percentage of success in few unimodal functions is in favour of particles that follow the classic updating formula as there is no need in these functions for high diversification. This proves the positive impact of the improved version to search precisely for better solutions as clarified in the following flowchart.

3.3.2.   Experiment 2 (Comparisons against variants of the BA)

In this experiment, The operational results of the HBNMA using several standard benchmark test functions with dimension $d = 10$ for all test functions except those of certain dimensions are compared with the standard BA and four variants of the BA to examine the ability of the proposed variant HBNMA in competing efficient different variants. The HBNMA in this comparison competes with the quantum annealing bat algorithm (QABA) ^[14], improved quantum annealing bat algorithm (IQBA) ^[25], group evolution hybrid bat algorithm (LMBA) ^[26], bat differential hybrid algorithm (BADE) ^[27], and the BA ^[5]. All algorithms involved in this experiment undergo the same conditions and common parameter values as stated in ^[14] and shown in Table 1 for fair comparison. The values of the other parameters related to a certain algorithm used in this comparison are found in their corresponding original literature. Following the same conditions that are stated by other established literature and yielding better results shows the adaptability and the efficiency of the HBNMA under any circumstances. Tables 5 and 6 show superior mean values (MV) compared to the BA and BADE in all test functions. In comparing with the LMBA, the HBNMA achieved the best MV in 13 of 15 functions, equal value in 1 function, and the worst value in 1 function. The competition was increased when comparing the HBNMA against the IQBA and QABA, but the superiority was in favor of the HBNMA. The HBNMA, with respect to the mean value, achieved the best values in 6 functions, the same in 8 functions, and the worst in only one function against the IQBA. In comparing with the QABA, the HBNMA recorded the best in 4, the same in 9, and the worst in 2. To statistically compare the mean results of the HBNMA against its counterparts, the Wilcoxon signed-rank test was used and is illustrated in Table 7. The p-value indicates whether or not there is a significant difference between the competing algorithms. Table 7 indicates that the HBNMA provided significantly better results than the IQBA, LMBA, BADE, and BA with $p < 0.05$. However, no significant difference was seen between the HBNMA and IQBA as $p > 0.05$. In terms of standard deviation (SD), the HBNMA recorded the best in 4, the same in 8, and the worst in 3 when comparing with the QABA. In comparing with the IQBA, the HBNMA had the best values in 11 functions, the same in 3, and the worst in 1. According to the LMBA and BADE, the HBNMA outperformed the SD results of them in most test functions. When comparing the HBNMA against the BA in SD, the superiority was in favor of the HBNMA. The similarity in some results in this comparison between the HBNMA and any other algorithm means that both algorithms achieved the optimum value. The low SD results reflected the stability of the HBNMA.

3.3.3.   Experiment 3 (comparisons against well-known heuristics utilizing IEEE CEC2014)

This experiment aims to show the efficiency of the proposed HBNMA using the 30 functions of the IEEE CEC2014 benchmark suite against state-of-the-art heuristics which are the genetic algorithm (GA) ^[1], particle swarm optimization (PSO) ^[2], differential evolution (DE) ^[28], and bat algorithm (BA) ^[5]. Each function for all competing algorithms is of dimension $d = 10$ that runs for 51 times independently with $d*10^5$ function evaluations for each run as a stopping criterion and a population size equal to 50 as shown in Table 1. Also, the values of the common parameters between the HBNMA and BA are stated in Table 1. The crossover and mutation probabilities of the GA are assigned as 0.9 and 0.05, respectively. In PSO, the inertia weight ranges from 0.9 to 0.4 and the coefficients of acceleration are of value 1.2. The parameters, that are used in DE, are the scale factor which belonging $[0.5, 0.9]$ and the crossover rate belonging to $[0.1, 0.9]$. Tables 8 and 9 illustrate the findings of the experiment in terms of MV and SD. The HBNMA succeeded to obtain the best mean error in 29 of 30 test functions and the same in 1 function when comparing with the GA and BA. In comparing with the PSO, the HBNMA obtained the best mean value in 21 of 30 test functions, the same value in 1 functions, and the worst in 8 functions. Furthermore, the HBNMA achieved the best mean value in 23 of 30 test functions, the same in 2 functions, and the worst in 5 functions when comparing against DE. The results showed the superiority of the HBNMA compared with all other algorithms in this experiment. This superiority is also translated in the p-value in Table 10 which demonstrated a significant difference between the HBNMA and the indicated algorithms.

The experimental results illustrated the superior behavior of the HBNMA in different test functions, which mimics real world problems. This is indicated in the following:

(1) In classical unimodal functions, there exists a single global optimum, but there are no local optima. So, these functions need fair intensification in the algorithm to be solved. The HBNMA succeeded easily in obtaining the global optimum for these functions because it has a good intensification ability as indicated in Experiments 1 and 2.

(2) The rotated unimodal functions, specifically functions F1–F3 from the CEC2014 benchmarking suit, are harder than the classical ones. So this suit of problems is perfect in evaluating the local searchability of the algorithm. The HBNMA obtained the best mean value in these functions relative to the state-of-the-art heuristics that are illustrated in Experiment 3 due to its strong local search.

(3) In classical multimodal functions, there are many local minima but only one global optimal. So, these functions are good to test the global searchability of any algorithm. The HBNMA behaved superiorly in these functions due to the strong diversification ability that is enhanced in this work as shown in Experiments 1 and 2.

(4) In shifted and rotated multimodal functions, specifically functions F4–F16 from the CEC2014 benchmarking suit, the HBNMA succeeded in obtaining the global optimum in the separable functions F8 and F10, but failed to obtain the global optimum in the other non-separable functions as shown in Experiment 3.

(5) In hybrid functions, F17–F22, which more closely approximate real-world benchmarks, the HBNMA obtained the best mean value in 4 of 6 when comparing against other algorithms. Furthermore, HBNMA has superior results with all composition functions, F23–F30, when comparing with the others.

4.   Time complexity

In this section, the time complexity of the chosen algorithms is discussed using big O notation. In the standard BA, the time complexity is O(n* MaxIt) ^[29], where n is the population size and MaxIt is the maximum number of iterations. The maximum number of iterations is taking as the stopping criterion in all algorithms in computing time complexity instead of the number of function evaluations for simplicity. In the proposed HBNMA, the population initialization process has a computational time of O(n). In the main optimization of the proposed algorithm, the calculation of the centroid of the simplex that is needed to compute the reflection and expansion steps is of time complexity O(n*d), where d is the dimension of the problem. In addition, each particle undergoes one of two updating processes which are constant time operations, O(1). Also, continuous expansions may occur as long as refining in the best solution found so far occurs. These continuous expansions have a time complexity of O(k), where k is the maximum number of improved expansions found. The value of the parameter k in the HBNMA is small because if there is an improved expansion, and the probability of obtaining another improved expansion sequentially is small. As a consequence, this probability decreases during further sequential expansions. So, this parameter can be neglected in the time complexity of the HBNMA despite its importance in the robustness of the algorithm. Therefore, the overall time complexity of the HBNMA is O(n*d*MaxIt). The time complexity of the DE algorithm, PSO algorithm, and GA is O(n*d*MaxIt) ^[30,31].

Hence, the proposed HBNMA has the same time complexity as in the state-of-the-art metaheuristics GA, PSO, and DE. In comparing with the BA in low-dimensional problems, the HBNMA has almost the same time complexity as in the BA. In high-dimensional problems, the HBNMA is more complex than the BA, but this does not affect the robustness of the HBNMA as it needs a lower number of iterations than that in the BA to reach the global optimal.

5.   Conclusions

In this work, a hybrid bat-Nelder-Mead algorithm (HBNMA) is presented to solve multidimensional, unconstrained optimization problems. This algorithm is based on introducing two steps of the NM in the updating formula of the particle's velocity of the standard BA. The reflection step is first introduced besides the standard updating formula of the BA. If an improvement in the fitness value is achieved, sequential expansion steps are then introduced to explore the area deeply, which accelerates the performance of the algorithm. Otherwise, the classic updating process, as in the standard BA, is performed. This mechanism enables particles to globally discover the search space and exploit any progress for deeper research for even better results. Accordingly, a satisfactory balance is achieved between global and local searches in the proposed algorithm. Finally, several numerical experiments were established and the simulation results of the proposed HBNMA showed superiority of the algorithm.

Future work

In future research, multi-objective and constrained optimization are going to be addressed as most real-world continuous problems are of that form. New techniques are going to be innovated to deal with the selection of solutions in multiobjective problems rather than the common used approaches. These new techniques are going to be integrated with the HBNMA and any other single-objective, continuous optimization algorithm to be able to handle complex multiobjective problems. Furthermore, greater emphasis can be devoted to create new techniques that can handle constraints in constrained optimization problems.

Use of Generative-AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Author contributions

Enas Suhail: Formal analysis, Original draft preparation, Visualization, Software; Mahmoud El-Alem: Investigation, Methodology, Writing—review and editing, Supervision; Omar Bazighifan: Resources, Writing—review and editing; Ahmed S. Zekri: Conceptualization, Validation, Writing—review and editing, Supervision. All authors have read and approved the final version of the manuscript for publication.

Conflict of interest

All authors declare that they have no conflicts of interest.

Supplementary

The following table describes the benchmark test functions used in Section 3.3: