Optimization problems are ubiquitous in engineering and scientific research, with a large number of such problems requiring resolution. Meta-heuristics offer a promising approach to solving optimization problems. The firefly algorithm (FA) is a swarm intelligence meta-heuristic that emulates the flickering patterns and behaviour of fireflies. Although FA has been significantly enhanced to improve its performance, it still exhibits certain deficiencies. To overcome these limitations, this study presents the Q-learning based on the adaptive logarithmic spiral-Levy flight firefly algorithm (QL-ADIFA). The Q-learning technique empowers the improved firefly algorithm to leverage the firefly's environmental awareness and memory while in flight, allowing further refinement of the enhanced firefly. Numerical experiments demonstrate that QL-ADIFA outperforms existing methods on 15 benchmark optimization functions and twelve engineering problems: cantilever arm design, pressure vessel design, three-bar truss design problem, and 9 constrained optimization problems in CEC2020.
Citation: Shuang Tan, Shangrui Zhao, Jinran Wu. QL-ADIFA: Hybrid optimization using Q-learning and an adaptive logarithmic spiral-levy firefly algorithm[J]. Mathematical Biosciences and Engineering, 2023, 20(8): 13542-13561. doi: 10.3934/mbe.2023604
Optimization problems are ubiquitous in engineering and scientific research, with a large number of such problems requiring resolution. Meta-heuristics offer a promising approach to solving optimization problems. The firefly algorithm (FA) is a swarm intelligence meta-heuristic that emulates the flickering patterns and behaviour of fireflies. Although FA has been significantly enhanced to improve its performance, it still exhibits certain deficiencies. To overcome these limitations, this study presents the Q-learning based on the adaptive logarithmic spiral-Levy flight firefly algorithm (QL-ADIFA). The Q-learning technique empowers the improved firefly algorithm to leverage the firefly's environmental awareness and memory while in flight, allowing further refinement of the enhanced firefly. Numerical experiments demonstrate that QL-ADIFA outperforms existing methods on 15 benchmark optimization functions and twelve engineering problems: cantilever arm design, pressure vessel design, three-bar truss design problem, and 9 constrained optimization problems in CEC2020.
| [1] |
A. M. Altabeeb, A. M. Mohsen, L. Abualigah, A. Ghallab, Solving capacitated vehicle routing problem using cooperative firefly algorithm, Appl. Soft Comput., 108 (2021), 107403. https://doi.org/10.1016/j.asoc.2021.107403 doi: 10.1016/j.asoc.2021.107403
|
| [2] |
M. H. Nadimi-Shahraki, H. Zamani, S. Mirjalili, Eqnhanced whale optimization algorithm for medical feature selection: A COVID-19 case study, Comput. Biol. Med., 148 (2022), 105858. https://doi.org/10.1016/j.compbiomed.2022.105858 doi: 10.1016/j.compbiomed.2022.105858
|
| [3] |
H. Zhang, Y. Shi, X. Yang, R. Zhou, A firefly algorithm modified support vector machine for the credit risk assessment of supply chain finance, Res. Int. Bus. Finance, 58 (2021), 101482. https://doi.org/10.1016/j.ribaf.2021.101482 doi: 10.1016/j.ribaf.2021.101482
|
| [4] |
J. S. Pan, L. G. Zhang, R. B. Wang, V. Snášel, S. C. Chu, Gannet optimization algorithm: A new metaheuristic algorithm for solving engineering optimization problems, Math. Comput. Simul., 202 (2022), 343–373. https://doi.org/10.1016/j.matcom.2022.06.007 doi: 10.1016/j.matcom.2022.06.007
|
| [5] |
G. Dhiman, K. K. Singh, M. Soni, A. Nagar, M. Dehghani, A. Slowik, et al., Mosoa: A new multi-objective seagull optimization algorithm, Expert Syst. Appl., 167 (2021), 114150. https://doi.org/10.1016/j.eswa.2020.114150 doi: 10.1016/j.eswa.2020.114150
|
| [6] | Z. Cui, X. Hou, H. Zhou, W. Lian, J. Wu, Modified slime mould algorithm via levy flight, in 2020 13th International Congress on Image and Signal Processing, Biomedical Engineering and Informatics (CISP-BMEI), IEEE, Public Library of Science San Francisco, USA, (2020), 1109–1113. https://doi.org/10.1109/cisp-bmei51763.2020.9263669 |
| [7] |
Y. Yang, Y. Gao, S. Tan, S. Zhao, J. Wu, S. Gao, et al., An opposition learning and spiral modelling based arithmetic optimization algorithm for global continuous optimization problems, Eng. Appl. Artif. Intell., 113 (2022), 104981. https://doi.org/10.1016/j.engappai.2022.104981 doi: 10.1016/j.engappai.2022.104981
|
| [8] | X. S. Yang, A. Slowik, Firefly algorithm, in Swarm Intelligence Algorithms, CRC Press, (2020), 163–174. https://doi.org/10.1201/9780429289071-12 |
| [9] |
N. Bacanin, T. Bezdan, K. Venkatachalam, F. Al-Turjman, Optimized convolutional neural network by firefly algorithm for magnetic resonance image classification of glioma brain tumor grade, J. Real Time Image Process., 18 (2021), 1085–1098. https://doi.org/10.1007/s11554-021-01106-x doi: 10.1007/s11554-021-01106-x
|
| [10] |
J. Zhang, Y. Huang, G. Ma, Y. Yuan, B. Nener, Automating the mixture design of lightweight foamed concrete using multi-objective firefly algorithm and support vector regression, Cem. Concr. Compos., 121 (2021), 104103. https://doi.org/10.1016/j.cemconcomp.2021.104103 doi: 10.1016/j.cemconcomp.2021.104103
|
| [11] |
M. Rigakis, D. Trachanatzi, M. Marinaki, Y. Marinakis, Tourist group itinerary design: When the firefly algorithm meets the n-person battle of sexes, Knowledge-Based Syst., 228 (2021), 107257. https://doi.org/10.1016/j.knosys.2021.107257 doi: 10.1016/j.knosys.2021.107257
|
| [12] |
A. Sharma, R. Chaturvedi, A. Bhargava, A novel opposition based improved firefly algorithm for multilevel image segmentation, Multimedia Tools. Appl., 81 (2022), 15521–15544. https://doi.org/10.1007/s11042-022-12303-6 doi: 10.1007/s11042-022-12303-6
|
| [13] |
V. Kumar, D. Kumar, A systematic review on firefly algorithm: past, present, and future, Arch. Comput. Methods Eng., 28 (2021), 3269–3291. https://doi.org/10.36227/techrxiv.12122748 doi: 10.36227/techrxiv.12122748
|
| [14] | X. S. Yang, Firefly algorithm, Lévy flights and global optimization, in Research and Development in Intelligent Systems XXVI, (2010), 209–218. https://doi.org/10.1007/978-1-84882-983-1_15 |
| [15] |
Q. X. Lieu, D. T. Do, J. Lee, An adaptive hybrid evolutionary firefly algorithm for shape and size optimization of truss structures with frequency constraints, Comput. Struct., 195 (2018), 99–112. https://doi.org/10.36227/techrxiv.12122748 doi: 10.36227/techrxiv.12122748
|
| [16] |
J. Wu, Y. G. Wang, K. Burrage, Y. C. Tian, B. Lawson, Z. Ding, An improved firefly algorithm for global continuous optimization problems, Expert Syst. Appl., 149 (2020), 113340. https://doi.org/10.1016/j.eswa.2020.113340 doi: 10.1016/j.eswa.2020.113340
|
| [17] |
B. A. Hassan, Cscf: a chaotic sine cosine firefly algorithm for practical application problems, Neural. Comput. Appl., 33 (2021), 7011–7030. https://doi.org/10.1007/s00521-020-05474-6 doi: 10.1007/s00521-020-05474-6
|
| [18] |
M. J. Goldanloo, F. S. Gharehchopogh, A hybrid obl-based firefly algorithm with symbiotic organisms search algorithm for solving continuous optimization problems, J. Supercomput., 78 (2022), 3998–4031. https://doi.org/10.1007/s11227-021-04015-9 doi: 10.1007/s11227-021-04015-9
|
| [19] |
S. Zhao, Y. Wu, S. Tan, J. Wu, Z. Cui, Y. G. Wang, Qqlmpa: A quasi-opposition learning and q-learning based marine predators algorithm, Expert Syst. Appl., 213 (2023), 119246. https://doi.org/10.1016/j.eswa.2022.119246 doi: 10.1016/j.eswa.2022.119246
|
| [20] |
T. N. Huynh, D. T. Do, J. Lee, Q-learning-based parameter control in differential evolution for structural optimization, Appl. Soft Comput., 107 (2021), 107464. https://doi.org/10.1016/j.asoc.2021.107464 doi: 10.1016/j.asoc.2021.107464
|
| [21] |
R. Qi, J. Q. Li, J. Wang, H. Jin, Y. Y. Han, Qmoea: A q-learning-based multiobjective evolutionary algorithm for solving time-dependent green vehicle routing problems with time windows, Inf. Sci., 608 (2022), 178–201. https://doi.org/10.1016/j.ins.2022.06.056 doi: 10.1016/j.ins.2022.06.056
|
| [22] |
B. Jang, M. Kim, G. Harerimana, J. W. Kim, Q-learning algorithms: A comprehensive classification and applications, IEEE Access, 7 (2019), 133653–133667. https://doi.org/10.1109/access.2019.2941229 doi: 10.1109/access.2019.2941229
|
| [23] |
R. H. Crites, A. G. Barto, Elevator group control using multiple reinforcement learning agents, Mach. Learn., 33 (1998), 235–262. https://doi.org/10.36227/techrxiv.21197626 doi: 10.36227/techrxiv.21197626
|
| [24] |
X. S. Yang, Firefly algorithm, stochastic test functions and design optimisation, Int. J. Bio-Inspired Comput., 2 (2010), 78–84. https://doi.org/10.1504/ijbic.2010.032124 doi: 10.1504/ijbic.2010.032124
|
| [25] |
A. M. Reynolds, M. A. Frye, Free-flight odor tracking in drosophila is consistent with an optimal intermittent scale-free search, PloS One, 2 (2007), e354. https://doi.org/10.1371/journal.pone.0000354 doi: 10.1371/journal.pone.0000354
|
| [26] |
V. A. Tucker, A. E. Tucker, K. Akers, J. H. Enderson, Curved flight paths and sideways vision in peregrine falcons (falco peregrinus), J. Exp. Biol., 203 (2000), 3755–3763. https://doi.org/10.1242/jeb.203.24.3755 doi: 10.1242/jeb.203.24.3755
|
| [27] |
I. Ahmadianfar, A. A. Heidari, S. Noshadian, H. Chen, A. H. Gandomi, Info: An efficient optimization algorithm based on weighted mean of vectors, Expert Syst. Appl., 195 (2022), 116516. https://doi.org/10.1016/j.eswa.2022.116516 doi: 10.1016/j.eswa.2022.116516
|
| [28] |
X. Tao, W. Guo, X. Li, Q. He, R. Liu, J. Zou, Fitness peak clustering based dynamic multi-swarm particle swarm optimization with enhanced learning strategy, Expert Syst. Appl., 191 (2022), 116301. https://doi.org/10.1016/j.eswa.2021.116301 doi: 10.1016/j.eswa.2021.116301
|
| [29] |
A. Kumar, G. Wu, M. Z. Ali, R. Mallipeddi, P. N. Suganthan, S. Das, A test-suite of non-convex constrained optimization problems from the real-world and some baseline results, Swarm Evol. Comput., 56 (2020), 100693. https://doi.org/10.1016/j.swevo.2020.100693 doi: 10.1016/j.swevo.2020.100693
|
Figures(3) / Tables(8)
Shuang Tan, Shangrui Zhao, Jinran Wu. QL-ADIFA: Hybrid optimization using Q-learning and an adaptive logarithmic spiral-levy firefly algorithm[J]. Mathematical Biosciences and Engineering, 2023, 20(8): 13542-13561. doi: 10.3934/mbe.2023604
DownLoad: