A survey on deep learning for financial risk prediction

1.   Introduction

In the past few decades, many meta-heuristic algorithms have been proposed to handle complex optimization problems, which are usually difficult to solve with traditional optimization methods relying on gradients. Meta-heuristic algorithms are problem-solving methods that mimic natural or abstract processes to efficiently explore solution spaces and find near-optimal solutions for complex optimization problems. Kennedy and Eberhart ^[1] developed particle swarm optimization algorithm (PSO), a population-based optimization algorithm inspired by social behavior and collective intelligence, where particles iteratively update their positions based on their own experience and the information from the best-performing particles in the swarm. Holland ^[2] proposed genetic algorithm (GA) that imitates the process of natural selection and genetic evolution to iteratively search for optimal solutions to complex problems. Wang et al. ^[3] proposed a monarch butterfly optimization algorithm (MBO) that imitates the migration behavior of monarch butterflies, where the search process is guided by a combination of exploration and exploitation strategies to find optimal solutions. Li et al. ^[4] presented a slime mold algorithm (SMA) that simulates the foraging behavior of slime mold to efficiently explore and exploit search spaces by forming networks of interconnected pathways. Wang ^[5] presented a moth search algorithm (MSA) that mimics the behavior of moths to navigate toward the optimal solution by adjusting their flight direction based on attraction and repulsion forces. Yang et al. ^[6] proposed a hunger games search (HGS) algorithm, where individuals compete for resources and survival. Ahmadianfar et al. ^[7] presented Runge-Kutta optimizer (RUN) that utilizes the logic of slope variations computed by the Runge-Kutta method to perform active exploration and exploitation for global optimization with an enhanced solution quality mechanism to avoid local optima and improve convergence speed. Tu et al. ^[8] proposed colony predation algorithm (CPA) that mimics the collective hunting behavior of predators to explore and exploit the solution space for optimization efficiently. Ahmadianfar et al. ^[9] presented INFO which utilizes the weighted mean concept, enhanced updating rules, vector combining and local search to optimize various problems. Heidari et al. ^[10] proposed Harris hawks optimization (HHO), which is inspired by the hunting behavior of Harris hawks, utilizing different strategies such as exploration, exploitation and cooperative hunting to solve optimization problems. Su et al. ^[11] proposed a RIME optimization algorithm, a nature-inspired metaheuristic algorithm based on the freezing process of water droplets in rime formation, utilizing dynamic freezing and melting mechanisms for efficient optimization. Compared with conventional algorithms, these meta-heuristic algorithms can handle non-differentiable, multimodal, hybrid and high-dimensional problems without requiring explicit gradient information.

After they are proposed, many researchers successfully solve complex optimization problems by using them in intelligent systems. Li et al. ^[12] introduced ACO-S, a bionic optimization algorithm-based dimension reduction method specifically designed for high-dimensional datasets like microarray data, demonstrating its ability to generate compact and informative gene subsets with high classification accuracy compared to existing bionic optimization algorithms. Thawkar et al. ^[13] presented a hybrid feature selection method, combining the butterfly optimization algorithm (BOA) and the ant lion optimizer (ALO) to effectively predict the benign or malignant status of breast tissue using mammogram images. Chakraborty et al. ^[14] developed a computational tool using a modified whale optimization method (WOA) to quickly and accurately determine the severity of COVID-19 illness through chest X-ray image analysis. Gao et al. ^[15] introduced a multi-objective optimization model and an improved genetic algorithm for cooperative mission assignment of heterogeneous UAVs, considering the balance between mission gains and UAV losses. Yu et al. ^[16] presented a multi-objective optimization problem and an improved genetic algorithm for cooperative mission planning of heterogeneous UAVs in cross-regional joint operations, considering makespan minimization, value expectation maximization, flexible base return and ammunition inventory constraints. Li et al. ^[17] investigated the scheduling problem of maintenance and repair tasks for carrier-based aircraft fleets in hangar bays, proposing a model and an improved teaching-learning-based optimization algorithm to enhance maintenance efficiency and reduce manual scheduling burden.

Among the algorithms mentioned above, PSO is popular because of its fast convergence, few parameters and easy programming implementation. However, the performance of the standard PSO depends mainly on its parameter settings ^[18] and different parameter settings may lead to different convergence outcomes. In addition, the update strategy of particles during the search process is mainly aimed at the global best, which leads the population to lose diversity in the population and increases the risk of being trapped in local optima ^[19,20,21]. To address these drawbacks and improve the performance of PSO, many PSO variants have been developed. Generally, research on PSO variants can be classified into three parts: parameter modification, population topology structure modification and novel learning strategy.

1) Parameter modification. PSO has some parameters such as inertia weight and acceleration coefficients. They have been modified in many ways. Shi and Eberhart ^[22] introduced the concept of inertia weight in PSO to improve velocity control and search effectiveness. Afterward, various adjustment strategies were advocated, including defining it as a time-varying function called global PSO (GPSO) ^[23]. Liu et al. ^[24] proposed a PSO algorithm with chaos inertia weight to enlarge the search range and increase population diversity. Chen et al. ^[25] proposed a hybrid PSO with sine cosine acceleration coefficients where the performance of PSO is improved by introducing sine and cosine acceleration coefficients. By using these strategies, the performance of PSO can be improved to some extent.

2) Population topology structure modification. In the standard PSO, the population size is fixed. A larger population size may increase computation costs, resulting in slow convergence, while a smaller size may converge in local optima. Some researchers have improved the performance of PSO by modifying the population topology structure. Kennedy and Mendes ^[26] proposed the local version of PSO (LPSO) with ring topology, in which different topologies based on neighborhood connections are used to maintain the diversity of the population. Liang and Suganthan ^[27] presented a dynamic multi-swarm PSO (DMS-PSO), in which the population comprises many small subgroups and a dynamic strategy is used to maintain population diversity.

3) Novel learning strategy. A well-designed learning strategy can enhance the efficiency and effectiveness of the PSO algorithm. Liang et al. ^[28] proposed the CLPSO, in which each particle updates its velocity by incorporating historical best information from different particles. This approach enables particles to acquire valuable information and improves their search behavior. Wang et al. ^[29] proposed a PSO-HLM algorithm that balances exploration and exploitation by utilizing a hybrid learning model and a multi-pools fusion strategy, improving convergence speed and avoiding local optima. Zhou et al. ^[30] proposed a LFIACL-PSO algorithm that enhances PSO by incorporating Lévy flight-based inverse learning, comprehensive learning strategy, ring-type topology and adaptive update of acceleration coefficients.

In this paper, an improved PSO combined with double-chaos search (DCS-PSO) is proposed. Chaotic dynamics is a branch of nonlinear dynamical systems. Chaos motion exhibits ergodicity, randomness and regularity properties and can traverse all states without repetition. For optimization problems, these properties of chaos can be used as a mechanism to avoid getting trapped in local optima during the search process. In DCS-PSO, we utilize two distinct chaotic systems to explore the search space globally and narrow it down, enabling PSO to focus on the vicinity of the optimal solution and alleviate its premature convergence caused by getting trapped in local optima. Additionally, we incorporate a chaotic motion search in parallel with the PSO search. If the solution obtained through the chaotic search outperforms the PSO's global best, it will be replaced with the solution from the chaotic motion to enhance population diversity. This strategy prevents all particles in the PSO from relying solely on the global optimal particle for learning, thereby facilitating escape from local optima.

The main contributions of this paper are described as follows.

1) Introducing a novel approach, DCS-PSO, which combines the strengths of PSO and chaotic dynamics.

2) Addressing the issue of premature convergence by narrowing down the search space using double-chaos search and enhancing population diversity through the logistic map, resulting in improved convergence accuracy and speed.

3) Conducting extensive experimental evaluations to demonstrate the effectiveness and superiority of DCS-PSO compared to standard PSO and other optimization algorithms commonly used in the field.

The rest of this paper is organized as follows. Section 2 introduces the related work. Section 3 describes the proposed DCS-PSO. Section 4 is the experimental studies. Conclusion and future work are provided in Section 5.

2.   Related works

2.1.   Standard PSO

PSO is an optimization algorithm based on swarm intelligence inspired by collective behavior in ecosystems such as bird foraging. In PSO, the problem is treated as an optimization problem of finding the optimal solution in a D-dimensional search space, where each particle $i$ in the population represents a possible solution and has two vectors: a position vector ${X_i} = \left[{{x_{i, 1}}, {x_{i, 2}}, \cdot \cdot \cdot, {x_{i, D}}} \right]$ and a velocity vector ${V_i} = \left[{{v_{i, 1}}, {v_{i, 2}}, \cdot \cdot \cdot, {v_{i, D}}} \right]$. In searching for the optimal solution, particles move and interact with each other in the search space. The particle $i$ adjusts its position based on the optimal position vector $\boldsymbol{pbes{t_i}} = \left[{pbes{t_{i, 1}}, pbes{t_{i, 2}}, \cdot \cdot \cdot pbes{t_{i, D}}} \right]$ it has reached in the past and the globally optimal position vector $\boldsymbol{gbest} = \left[{gbes{t_1}, gbes{t_2}, \cdot \cdot \cdot, gbes{t_D}} \right]$ discovered by the entire swarm. PSO attempts to find a balance between local search and global search in this way. The update rules for the velocity and position of particle $i$ in the $d$ th dimension are as follows:

where ${c_1}$ and ${c_2}$ denote the acceleration coefficients, usually set to 2, ${r_1}$ and ${r_2}$ are two uniformly distributed values in the range [0, 1]. The parameters ${V_{\min }}$ and ${V_{\max }}$ can be used to determine an appropriate range of velocity values for each particle to prevent it from wandering too far outside the search space.

The global minimum optimization problem (2.3) for a D-dimensional continuous object can be described as:

The standard PSO can be described in the following steps:

Step 1: Initialize $N$ particles. For each particle $i$ in the population:

Step 1.1: Initialize ${X_i}$ randomly in the range $\left[{\boldsymbol a,\boldsymbol b} \right]$.

Step 1.2: Initialize ${V_i}$ randomly in the range $\left[{{V_{\min }}, {V_{\max }}} \right]$.

Step 1.3: Evaluate the fitness value ${f_i}$.

Step 1.4: Set $\boldsymbol{pbes{t_i}}$ as the particle's current position and $gbest$ as the particle with the best fitness value in the population.

Step 2: Loop until the stop condition is satisfied:

Step 2.1: Update the velocity ${V_i}$ and position ${X_i}$ for each particle $i$ according to Eqs (2.1) and (2.2).

Step 2.2: Evaluate the fitness value ${f_i}$ of each particle $i$.

Step 2.3: Update $\boldsymbol{pbes{t_i}} ~~ if~~ {\text{ }}{f_i} < {f_\boldsymbol{{pbes{t_i}}}}, \forall i \leqslant N$ for each particle $i$.

Step 2.4: Update the global optimal position $\boldsymbol{gbest}~~ ~~{f_\boldsymbol{{gbest}}} \leqslant {f_i}, {\text{ }}\forall i \leqslant N$.

2.2.   Chaos algorithms

2.2.1.   Chaos optimization algorithm

Chaos is one of the most exciting properties exhibited by the brain ^[31]. It is the phenomenon of complex, unpredictable and random-like behavior arising from simple deterministic nonlinear systems rather than a state of disorder ^[32]. Researchers have recently developed many optimization algorithms based on chaos theory to address complex optimization problems ^[33,34,35].

COA was first proposed by Li and Jiang ^[32,35] in 1997. Its core idea is to introduce the chaotic state into optimization variables via the carrier wave method, map the traversal range of chaos motion to the value range of optimization variables and then search for the global optimal solution using chaotic dynamics instead of random search. The chaotic dynamics equation chosen in COA is the Logistic map ^[36], which is defined as follows:

where $\mu$ is the control parameter, $n = 0, 1, 2, ...$. Although the above equation is definite, it exhibits chaotic dynamics when $\mu = 4$ and ${x^{\left(0 \right)}} \notin \left\{ {0, 0.25, 0.5, 0.75, 1} \right\}$, the property that a minute difference in the initial value of the chaotic variable would lead to significant differences in output. This is the sensitive dependence on initial conditions, the basic characteristic of chaos. This property enables the trajectory of chaotic variables to traverse the entire search space. The process of COA can be defined through the following equation.

where $c{x_d}$ is the $d$ th chaotic variable and $k$ represents the number of iterations. Obviously, $cx_d^{\left(k \right)}$ is distributed in the range (0, 1) when the initial $cx_d^{\left(0 \right)} \in \left({0, 1} \right)$ and $cx_d^{\left(0 \right)} \notin \left\{ {0.25, 0.5, 0.75} \right\}$.

For the optimization problem (2.3), the procedures of COA are as follows.

Step 1: Set $k = 0$, and initialize $D$ chaotic variables $cx_d^{\left(0 \right)}$ in the range (0, 1) with very small differences. The current optimal function value ${f^ * }$ is initialized to a large value, with ${X}^{\ast } = \left[{x}_{1}^{\ast }, {x}_{2}^{\ast }, \cdot \cdot \cdot {x}_{D}^{\ast }\right]$ representing the corresponding optimal variables.

Step 2: Map the $D$ chaotic variables $c{x_d}$ to the $D$ optimization variables ${x_d}$ of the optimization problem (2.3) through the carrier wave method of Eq (2.6).

Step 3: Iterative search using chaotic variables.

If $f\left({{X^{\left(k \right)}}} \right) \leqslant {f^ * }$, then ${f^ * } = f\left({{X^{\left(k \right)}}} \right), {\text{ }}{X^ * } = {X^{\left(k \right)}}$. Else if $f\left({{X^{\left(k \right)}}} \right) > {f^ * }$, then give up ${X^{\left(k \right)}}$.

Step 4: Let $k = k + 1$ and continue the chaotic traversal through Eq (2.5).

Step 5: Repeat steps 2 to 4. If the best function value ${f^ * }$ remains unchanged after several iterations, the second carrier wave is applied according to Eq (2.7).

where $x_d^ *$ is the current optimal solution, $\alpha$ is an adjusting constant which can be less than 1 and $\alpha \cdot cx_d^{\left({k + 1} \right)}$ generates $i$ chaotic states with small ergodic ranges around $x_d^ *$.

Step 6: Continue the iterative search using the chaotic variables after the second carrier wave until the stop criterion is satisfied.

2.2.2.   Some representative variants of COA

As a simple and efficient optimization algorithm, COA has garnered the attention of many researchers for further improvement. Its primary disadvantage is that the sensitive dependence of chaotic motion on the initial values will cause the algorithm's instability and a large number of traversal searches are required before the search space is reduced. Moreover, the lack of an explicit search stop condition results in the need for parameter tuning according to specific problems, limiting the algorithm's generality. Therefore, many variants of COA aimed at these problems have emerged.

Xiu et al. ^[37] proposed a double-chaos optimization algorithm (DCOA) that uses two different chaotic systems to explore the search space independently and in parallel and narrows down the search space when the distance between the optimal positions of the two systems satisfies specific criteria. Therefore, DCOA overcomes some of the deficiencies of COA. Additionally, DCOA provides a condition for narrowing the search space, which improves the algorithm's generality, avoids multiple blind searches and reduces its running cost. Liang and Gu ^[38] developed a chaos optimization algorithm based on parallel computing (PCOA), which performs parallel searches using several different sets of chaotic variables, effectively reducing the sensitive dependence of chaotic motion on initial values.

3.   Proposed method

Based on in-depth research of the chaos algorithms and the PSO algorithm, we combined the advantages of the two types of algorithms and proposed DCS-PSO. The details are as follows:

3.1.   Space reduction based on double-chaos search mechanism

Although a single chaos search mechanism is easy to implement and can avoid being trapped in local optima, it often requires a large number of iterations to narrow down the search space to ensure that the optimal solution is within the narrowed area.

Additionally, the stop condition of the search needs to be adjusted based on the specific problem, limiting the algorithm's generality. In contrast, the DCOA determines the requirement for narrowing the space based on the distance between the optimal positions found by each chaotic system, which improves the algorithm's generality. However, there is a risk that the algorithm may continuously narrow down the search space, making it difficult to converge accurately to a point and potentially excluding the global optimal solution from the search space. Nonetheless, experimental verification (see Section 4.1) suggests that the double-chaos search mechanism can narrow the search space to the neighborhood of the global optimum after only one sufficient search.

In the double-chaos search mechanism, we select two chaotic systems: The logistic map and the tent map. The orbit points of the logistic map are distributed near the edges, which to some extent can explain why chaotic motion has the advantage of escaping from local optima ^[39]. However, several breakpoints $\left\{ {0, 0.25, 0.5, 0.75, 1} \right\}$ in the logistic map make it difficult to traverse specific chaos states during the search process. The orbit points of the tent map are distributed more evenly. Its equation is defined in Eq (3.1). Figure 1(a) and (b) show the scatter plots of the two chaos maps for 2000 iterations, respectively and (c) and (d) show their chaotic dynamics.

where $\beta = 0.4$.

The double-chaos search mechanism is illustrated in Figure 2. In the search space A, two chaotic systems $\boldsymbol{cx}$ and $\boldsymbol{cy}$ are used to perform independent parallel searches. When the distance between the optimal solutions $\boldsymbol c{\boldsymbol x^ * }$ and $\boldsymbol c{\boldsymbol y^ * }$ found by $cx$ and $cy$ is small enough, namely,

where ${X^ * }$ and ${Y^ * }$ represent the corresponding optimization variables, $\gamma \in (0, 0.25]$ and is set to 0.15 in this study, the search space is narrowed down from A to B according to Eqs (3.3) and (3.4).

where $\xi \in \left[{1, 2} \right]$ and it is set to 1.5 in this study.

3.2.   An improved particle swarm optimization combined with double-chaos search

Based on the chaos search mechanism, we propose a two-stage iteration strategy named DCS-PSO. The algorithm employs the double-chaos search mechanism for global exploration and narrowing the search space and then uses the improved PSO for further fine search in the narrowed search space.

For the D-dimensional optimization problem (2.3), the detailed steps of the first stage of DCS-PSO are listed in Algorithm 1:

The flowchart of the first stage of DCS-PSO is shown in Figure 3(a).

In conventional PSO, all particles learn from the global optimal particle in the population to update their positions and velocities. This mechanism is advantageous in achieving fast convergence, but it also leads to the need for more diversity and often causes premature convergence due to falling into local optimum ^[40]. The conventional PSO may require longer to find the optimal solution when dealing with multimodal optimization problems with a large search space.

Although the risk of being trapped in local optima can be reduced by narrowing the search space, PSO may still fall into a local optimum in the reduced search space. To enhance the population's diversity, in the narrowed search space, we perform a chaos search on the best solution found by the two chaotic systems using the logistic map. The best solution found by the logistic map and the population guides the algorithm to converge toward the global optimum. The velocity of particle $i$ on dimension $d$ is updated according to the following rules.

where $best$ denotes the best solution between chaos search and population search, $f\left({cbest} \right)$ is the optimal fitness value found by chaos search and $cbest$ is the corresponding solution.

The detailed steps of the second stage of DCS-PSO are listed in Algorithm 2:

The flowchart of the second stage of DCS-PSO is shown in Figure 3(b).

It is worth noting that many researchers have attempted to introduce the chaos search mechanism into intelligent optimization algorithms ^[39,41]. The most common strategy is to use chaotic local search (CLS) after each iteration to conduct a fine search near the current optimum. Although this approach improves the convergence accuracy of the algorithm, it needs to address the problem of the population falling into local optima. Therefore, in addition to CLS, other strategies are often required to maintain the diversity of the population. However, some of these strategies may sacrifice the performance of convergence speed.

4.   Experimental studies

In this chapter, simulation experiments are conducted to validate the performance of DCS-PSO. We first conduct experiments using six 2-dimensional benchmark functions to validate the effectiveness of the DCS-PSO search space narrowing mechanism. After that, the performance of the DCS-PSO algorithm is compared with five typical PSO variants placed on ten benchmark functions in terms of solution accuracy, statistical significance test and convergence speed. Finally, to verify the performance of DCS-PSO in complex and multimodal conditions, an authoritative test suite CEC2017 is selected in this paper.

All simulation experiments were conducted on the same PC. The configuration information of the PC is shown in Table 1.

4.1.   Experiments on DCS-PSO and other chaos algorithms

4.1.1.   Six 2-Dim benchmark functions

To validate the effectiveness of the search space narrowing mechanism in DCS-PSO, we conducted experiments using six 2-dimensional benchmark functions ^[38,42]. DCS-PSO was compared with other chaos-based algorithms, namely COA, DCOA and PCOA. The narrowed search space can be visually observed through the contour map of the 2-dimensional function. The six benchmark functions are listed below.

(1) Jong function:

The function has a global minimum of 0 at (1, 1).

(2) Camel function:

The function has a global minimum of −1.031628 at two points (−0.0898, 0.7126) and (0.0898, −0.7126).

(3) GP-Goldstein-Price:

The function has a global minimum of 3 at (0, −1) and there are four local minima in the minimization region.

(4) BR-Branin:

The global minimum of this function is approximately equal to 0.398 at three points (−3.142, 12.275), (3.142, 2.275) and (9.425, 2.425).

(5) RA-Rastrigin:

The function has a global minimum of −2 at (0, 0) and its lattice structure has about 50 local minima.

(6) SH-Shuber:

The function has 760 local minima, 18 of which are global with a value of −186.7309.

The RA-Rastrigin function and the SH-Shuber function are relatively complex and have a large number of local minima, which can effectively test the algorithm's ability to escape from the local optima. Their 3D stereograms are shown in Figure 4.

4.1.2.   Experimental strategies

To ensure the fairness of the experimental results, each algorithm will run on each benchmark function independently 50 times and the mean execution time, the mean optimal values and the standard deviations will be provided. The specific parameter settings of all algorithms are listed in Table 2.

4.1.3.   Results and discussion

Table 3 shows the execution time (E.T.), the mean optimal values (Mean) and standard deviations (S.D.) of COA, DCOA, PCOA and DCS-PSO. The best results of each algorithm on each test function are indicated in bold and the second-best results are marked with an underline.

Based on the mean values in Table 3, the proposed DCS-PSO outperforms the other three chaos-based methods. Compared with COA, DCOA and PCOA, DCS-PSO achieves superior results on five functions (${f_J}$, ${f_C}$, ${f_G}$, ${f_R}$ and ${f_S}$). COA performs well in solving simple optimization problems and acquires the theoretically optimal solutions on both functions ${f_J}$ and ${f_C}$. However, it performs poorly on function ${f_S}$, which has multiple local minima, possibly due to the challenge of attaining certain chaos states in a large search space. The convergence accuracy of DCOA is relatively poor. As mentioned earlier, DCOA may exclude the optimal solution while continuously narrowing the search space. However, compared with COA and PCOA, DCOA shows a shorter convergence time, which reflects its efficiency in narrowing the search space. The overall performance of PCOA is comparable to COA and it is the only algorithm that converges to the global optimum on function ${f_B}$. DCS-PSO has a faster convergence speed due to the combination of double-chaos search and PSO. The S.D. values in Table 3 show that DCS-PSO has the smallest S.D. values among the six benchmark functions, indicating that DCS-PSO is more stable and reliable.

Figure 5(a) and (b) show the contour plots of functions ${f_R}$ and ${f_S}$ projected onto the XOY plane, respectively. The red rectangle in each plot represents the narrowed search space, demonstrating the ability of DCS-PSO to reduce the search space of complex optimization problems. These plots visually display that DCS-PSO can accurately converge to the vicinity of the global optimum, validating the effectiveness of the proposed method.

4.2.   Experiments on DCS-PSO and other PSO variants

In this subsection, five representative PSO variants are selected for comparison. The first one is the PSO with inertia weight (PSO-w) ^[22], which can better control the search process and is characteristic of fast convergence when dealing with unimodal problems. The second one is the global version of PSO (GPSO) ^[23], whose inertia weight is decreased linearly from 0.9 to 0.4, improving the population's global search ability. The third one is the local version of PSO (LPSO) ^[26] with ring topology, in which different topologies based on neighborhood connections are used to maintain the diversity of the population. The fourth one is the Comprehensive Learning PSO (CLPSO) ^[28], in which each particle updates its velocity by learning the historical best information of different particles. The fifth is the dynamic multi-swarm PSO (DMS-PSO) ^[27], in which the population comprises many small subgroups and a dynamic strategy is used to maintain population diversity. CLPSO and DMS-PSO are designed to improve the performance of PSO on multimodal problems.

4.2.1.   Benchmark functions

To investigate the performance of DCS-PSO on different optimization problems, ten benchmark functions are tested. Based on the properties of these functions, they are divided into two groups. The first group consists of 5 unimodal benchmark functions with only one global optimum value, mainly used to test the convergence accuracy and speed. The second group consists of 5 multimodal benchmark functions used to test the algorithm's global search ability and its ability to escape from local optima. These benchmark functions are widely used in the literature ^[25,43,44]. The details of these benchmarks are shown in Table 4.

In the first group, the global optimal position of function ${f_4}$ is ${\left[1 \right]^D}$.The global optimal position of the remaining four functions is ${\left[0 \right]^D}$ and the theoretical optimal values of the five functions are 0. ${f_1}$ is the sphere function and is easy to solve. ${f_2}$ and ${f_3}$ are the Schwefel functions. ${f_4}$ is the Rosenbrock function, it becomes a simple multimodal problem when its dimension is more than three and has a narrow gap between the perceived local optima and the global optimum. ${f_5}$ is the noise function, which is challenging to find the global optimum because there is a perturbation factor.

In the second group, the global optimal positions of all functions are ${\left[0 \right]^D}$ and the global optimal values are 0. Rastrigin's function (${f_6}$) has a large number of local optima. ${f_7}$ has a narrow global basin and a large number of local optima. ${f_8}$ is the Griewank's function. ${f_9}$ and ${f_{10}}$ are the penalized functions.

In ${f_9}$, the ${y_i}$ is updated by ${y_i} = 1 + \frac{1}{4}\left({{x_i} + 1} \right)$.

In ${f_9}$ and ${f_{10}}$, $u\left({{x_i}, a, k, m} \right) = \left\{ \begin{array}{l} k{\left({{x_i} - a} \right)^m}, {\text{ }}{x_i} > a \hfill \\ {\text{ }}0, {\text{ }} - a \leqslant {x_i} \leqslant a \hfill \\ k{\left({ - {x_i} - a} \right)^m}, {\text{ }}{x_i} < - a \hfill \\ \end{array} \right.$.

4.2.2.   Experimental strategies

In the experiments, a population size of 40 and a maximum number of iterations of 2000 were used for all algorithms. The parameters for the five PSO variants were kept consistent with their respective literature. The benchmark functions were tested with dimensionalities of 10, 30 and 50 and each benchmark function was independently run 50 times in different dimensions to ensure the fairness of the experimental results. The error between the best optimum found by the algorithm and the actual global optimum was recorded and the mean error and standard deviation were displayed. The specific parameter settings for all algorithms can be found in Table 5.

Furthermore, the nonparametric Wilcoxon rank-sum tests were executed for each benchmark function to determine whether the results obtained by DCS-PSO are significantly different from the best results achieved by the other algorithms

4.2.3.   Results and discussion

Tables 6, 7 and 8 display the means (Mean) and standard deviations (S.D.) of six algorithms on 10 test functions with dimensions of 10, 30 and 50, respectively. The k value presented in the second column of each table is the outcome of the nonparametric Wilcoxon rank-sum tests on our proposed algorithm and the second-best algorithms (indicated with underlining in tables). A k value of 1 indicates a significant difference in the performances of the two algorithms with 95% certainty, while a k value of 0 suggests no significant difference. Moreover, Table 7 ranks the algorithms according to their mean solution accuracy. The best results for each benchmark function are denoted in bold and the second-best results are underlined.

1) Results for the 10-D problems: For ${f_1}$, the performance of DCS-PSO is second only to LPSO, which maintains population diversity based on different neighborhood topologies, so its performance varies greatly on problems with different topologies. For ${f_2}$ and ${f_3}$, our DCS-PSO very clearly outperforms all other algorithms, in solution accuracy. DCS-PSO achieves comparable result to other algorithms on ${f_4}$ with CLPSO and DMS-PSO exceeding it. ${f_5}$ is a noise function that is difficult to solve due to the perturbation factor. However, DCS-PSO still gets the best result on this problem.

For multimodal problems, comparing DCS-PSO with CLPSO and DMS-PSO, which are algorithms proposed to improve the performance of PSO on multimodal problems, can demonstrate the competitiveness of DCS-PSO. DCS-PSO and CLPSO perform equally well with zero error on Rastrigin's function (${f_6}$), which has many local optima. For ${f_7}$, DCS-PSO achieves the best result, but according to the statistical test result, its performance is comparable to that of DMS-PSO. For ${f_8}$, PSO-w, GPSO, DMS-PSO and DCS-PSO perform similarly, but CLPSO performs relatively better. For two penalized functions ${f_9}$ and ${f_{10}}$, although the performance of DCS-PSO on ${f_9}$ is not as good as that of GPSO, CLPSO and DMS-PSO, DCS-PSO achieves the lowest error on ${f_{10}}$.

Overall, DCS-PSO obtains the best results in solving three unimodal functions (${f_2}$, ${f_3}$ and ${f_5}$) and three multimodal functions (${f_6}$, ${f_7}$ and ${f_{10}}$). Additionally, Table 6 shows that the S.D. values of DCS-PSO on most test functions are much smaller than those of other algorithms, indicating that DCS-PSO is more stable during the convergence process.

2) Results for the 30-D and 50-D problems: The experiments conducted on 10-D problems are repeated for 30-D and 50-D problems and the results are presented in Tables 7 and 8. All algorithms exhibited similar characteristics as on the 10-D problems. However, high dimensionality leads to increased complexity, resulting in a decline in the performance of optimization algorithms on most problems. As shown in Tables 7 and 8, for ${f_1}$, ${f_2}$ and ${f_5}$ with 30 and 50 dimensions, DCS-PSO consistently outperforms all other algorithms, particularly on ${f_2}$. Notably, for the Rosenbrock function (${f_4}$), DCS-PSO performs similarly to other algorithms in 10 dimensionalities. However, its performance does not deteriorate as the dimension increases, illustrating that high dimensionality does not significantly decrease the performance of DCS-PSO on this problem.

For ${f_6}$ with 30 and 50 dimensions, CLPSO always performs best among all algorithms. DCS-PSO is still best on ${f_7}$ with 30 dimensionalities, while GPSO performs better on this problem with 50 dimensions. CLPSO and DMS-PSO retain their superiority on ${f_8}$ and ${f_9}$ with 30 and 50 dimensionalities, respectively. However, the nonparametric Wilcoxon rank-sum test results indicate that DCS-PSO is not statistically significantly poorer than CLPSO and DMS-PSO. DCS-PSO remains the best-performing algorithm on ${f_{10}}$, which suggests that its performance is not affected by the increased dimensionality of this function.

3) Comparison of convergence speed: Figure 6 shows the convergence characteristics in terms of the error between the best fitness value found and the actual best function value of the median run of each algorithm for each test function with 30 dimensions. Since the convergence characteristics for the 10-D and 50-D cases are similar to those in the 30-D case, they are not presented.

Combining the numerical results and convergence plots, it can be concluded that all algorithms perform well in solving ${f_1}$ and ${f_2}$ with three different dimensionalities. However, DCS-PSO is faster than other algorithms. PSO exhibits similar convergence characteristics as DCS-PSO on ${f_2}$. The convergence characteristics of each algorithm on ${f_1}$ and ${f_2}$ are shown in Figure 6(a) and (b). Figure 6(c) indicates that CLPSO has better convergence speed and accuracy than other algorithms on ${f_3}$. Although DCS-PSO has slightly lower convergence accuracy than CLPSO on ${f_4}$, it has the fastest convergence speed as illustrated in Figure 6(d). All algorithms perform similarly on the noise problem (${f_5}$). Figure 6(e) displays that these algorithms show a slow decrease in error, but DCS-PSO has the fastest convergence speed and the lowest error.

The advantages of CLPSO and DMS-PSO are revealed through some multimodal problems. As shown in Figure 6(f), although the convergence speed is slow, CLPSO has the best search accuracy on ${f_6}$. In contrast, other algorithms cannot continue to search effectively after reaching a certain accuracy. From the convergence characteristics in Figure 6(g), the performances of DCS-PSO and DMS-PSO are also excellent on ${f_7}$, but DCS-PSO has the fastest convergence speed. Although DCS-PSO is not as good as CLPSO and DMS-PSO in terms of average errors on ${f_8}$ and ${f_9}$, there is no statistically significant difference according to the Wilcoxon tests, as verified in Figure 6(h, i). It can be seen from Figure 6(j) that DCS-PSO has the best search efficiency on ${f_{10}}$. Considering the numerical results and the convergence speeds, DCS-PSO performs best among all tested algorithms.

4) Discussion: The nonparametric Wilcoxon rank-sum tests indicate that DCS-PSO is significantly better than or equivalent to the other algorithms on most problems with three different dimensionality cases. When the problems are 10-dimensional, DCS-PSO is significantly better than other algorithms on ${f_2}$, ${f_3}$, ${f_5}$ and ${f_{10}}$, and is equivalent to CLPSO and DMS-PSO on ${f_6}$ and ${f_7}$, respectively. For ${f_8}$ and ${f_9}$, with 30 and 50 dimensionalities, DCS-PSO performs equivalently to CLPSO and DMS-PSO, respectively. All statistical results demonstrate that DCS-PSO performs very well overall.

In the above experiments, DCS-PSO shows better convergence speed and accuracy on most unimodal problems, indicating that it is more effective in solving such problems. Simultaneously,

the performance of DCS-PSO on most multimodal problems is comparable to that of CLPSO and DMS-PSO, demonstrating its ideal global search performance and ability to escape local optima.

4.3.   Experiments on CEC2017

This section further uses a widely recognized CEC2017 test suite to confirm the performance of DCS-PSO with PSO variants. In CEC2017, there are 29 functions (F2 has been excluded because it shows unstable behavior, especially for higher dimensions) which can be classified into four categories: unimodal functions (${f_1}$, ${f_3}$), multimodal functions (${f_4}$ – ${f_{10}}$), hybrid functions (${f_{11}}$ – ${f_{20}}$) and composition functions (${f_{21}}$ – ${f_{30}}$). Due to space constraints, we only select D = 30 for analysis and discussion. The best value for each function is shown in bold.

4.3.1.   Results comparison

Table 9 shows that CLPSO and DMS-PSO obtain the best result in unimodal functions, followed by DCS-PSO. In multimodal functions, DCS-PSO achieves the best performance on ${f_4}$, ${f_5}$, ${f_8}$ and ${f_9}$, 4 of 7 multimodal functions in terms of mean value, followed by GPSO, CLPSO, and DMS-PSO, which all obtain the best solutions on one function.

For the 10 hybrid functions, DCS-PSO reaches the best solutions on ${f_{13}}$, ${f_{14}}$, ${f_{15}}$ and ${f_{18}}$, 4 of 10 hybrid functions. DMS-PSO also performs remarkably since it reaches the best solutions on 3 out of 10 functions, followed by GPSO, LPSO and CLPSO, which all obtain the best solutions on one function. In the ten composition functions, DMS-PSO attains remarkable results on ${f_{21}}$, ${f_{22}}$, ${f_{23}}$, ${f_{24}}$, ${f_{26}}$ and ${f_{27}}$, 6 out of 10 composition functions. The comparison results indicate that DCS-PSO works well in complex and multimodal situations.

4.3.2.   Friedman test results

Friedman test is used in this section to compare the performance of DCS-PSO and other PSO variants. The results of the Friedman test are shown in Table 10. It can be seen from Table 10 that DCS-PSO achieves a remarkable result on the CEC2017, followed by DMS-PSO, PSO-w, GPSO, CLPSO and LPSO. The overall performance in CEC2017 reveals that DCS-PSO has a distinct advantage over other compared algorithms.

5.   Conclusions and future works

In this paper, we propose an improved particle swarm optimization combined with double-chaos search mechanism, namely DCS-PSO. The standard PSO is simple and efficient, but it is difficult to maintain population diversity when dealing with complex problems with a large search space, leading to premature convergence and getting trapped in local optima. The chaos search mechanism has the advantages of global traversal and avoiding falling into local optima.

In the first stage of DCS-PSO, we adopt the double-chaos search mechanism to narrow the search space to the vicinity of the optimal solution, effectively reducing the risk of PSO getting trapped in local optima. In the second stage, to enhance the population diversity, the logistic map is utilized to perform a global search in the narrowed search space and the best solution obtained from both chaos search and population search will guide the population to converge. Experimental studies show that the algorithm can effectively narrow the search space and has better convergence accuracy and speed for most functions.

Of course, according to the no free lunch theorem, DCS-PSO cannot optimally solve every kind of global optimization problem. Although we use the chaos search mechanism to improve the convergence speed and ability to escape the local optima of DCS-PSO, it also brings about a problem worth further investigation: how to ensure that the optimal solution must exist in the narrowed search space. In the future, we will continue conducting in-depth research on chaos theory, optimize our DCS-PSO algorithm and try to apply DCS-PSO to solving these real-world problems.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (Grant No 82260849); the National Natural Science Foundation of China (Grant No 61562045); and Jiangxi University of Chinese Medicine Science and Technology Innovation Team Development Program (Grant No CXTD22015).