Teaching learning-based whale optimization algorithm for multi-layer perceptron neural network training

Yongquan Zhou; Yanbiao Niu; Qifang Luo; Ming Jiang; Yongquan Zhou; Yanbiao Niu; Qifang Luo; Ming Jiang

doi:10.3934/mbe.2020319

Mathematical Biosciences and Engineering

2020, Volume 17, Issue 5: 5987-6025. doi: 10.3934/mbe.2020319

Previous Article Next Article

Research article Special Issues

Teaching learning-based whale optimization algorithm for multi-layer perceptron neural network training

1.
College of Artificial Intelligence, Guangxi University for Nationalities, Nanning 530006, China
2.
Key Laboratory of Guangxi High Schools Complex System and Computational Intelligence, Nanning 530006, China
3.
Guangxi Key Laboratories of Hybrid Computation and IC Design Analysis, Nanning 530006, China
4.
Guangxi Institute of Digital Technology, Nanning 530000, China

Received: 29 May 2020 Accepted: 24 August 2020 Published: 10 September 2020

This paper presents an improved teaching learning-based whale optimization algorithm (TSWOA) used the simplex method. First of all, the combination of WOA algorithm and teaching learning-based algorithm not only achieves a better balance between exploration and exploitation of WOA, but also makes whales have self-learning ability from the biological background, and greatly enriches the theory of the original WOA algorithm. Secondly, the WOA algorithm adds the simplex method to optimize the current worst unit, averting the agents to search at the boundary, and increasing the convergence accuracy and speed of the algorithm. To evaluate the performance of the improved algorithm, the TSWOA algorithm is employed to train the multi-layer perceptron (MLP) neural network. It is a difficult thing to propose a well-pleasing and valid algorithm to optimize the multi-layer perceptron neural network. Fifteen different data sets were selected from the UCI machine learning knowledge and the statistical results were compared with GOA, GSO, SSO, FPA, GA and WOA, severally. The statistical results display that better performance of TSWOA compared to WOA and several well-established algorithms for training multi-layer perceptron neural networks.
- teaching learning-based,
- whale optimization algorithm,
- multi-layer perceptron (MLP) neural network,
- metaheuristic algorithm
Citation: Yongquan Zhou, Yanbiao Niu, Qifang Luo, Ming Jiang. Teaching learning-based whale optimization algorithm for multi-layer perceptron neural network training[J]. Mathematical Biosciences and Engineering, 2020, 17(5): 5987-6025. doi: 10.3934/mbe.2020319

Related Papers:

Abstract

This paper presents an improved teaching learning-based whale optimization algorithm (TSWOA) used the simplex method. First of all, the combination of WOA algorithm and teaching learning-based algorithm not only achieves a better balance between exploration and exploitation of WOA, but also makes whales have self-learning ability from the biological background, and greatly enriches the theory of the original WOA algorithm. Secondly, the WOA algorithm adds the simplex method to optimize the current worst unit, averting the agents to search at the boundary, and increasing the convergence accuracy and speed of the algorithm. To evaluate the performance of the improved algorithm, the TSWOA algorithm is employed to train the multi-layer perceptron (MLP) neural network. It is a difficult thing to propose a well-pleasing and valid algorithm to optimize the multi-layer perceptron neural network. Fifteen different data sets were selected from the UCI machine learning knowledge and the statistical results were compared with GOA, GSO, SSO, FPA, GA and WOA, severally. The statistical results display that better performance of TSWOA compared to WOA and several well-established algorithms for training multi-layer perceptron neural networks.

References

[1]	W. S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys., 5 (1943), 115-133.
[2]	A. Krogh, What are artificial neural networks?, Nat. Biotechnol., 26 (2008), 195-197.
[3]	A. Leśnia, M. Juszczyk, Prediction of site overhead costs with the use of artificial neural network based model, Arch. Civil. Mech. Eng., 18 (2018), 973-982.
[4]	S. Kaymak, A. Helwan, D. Uzun, Breast cancer image classification using artificial neural networks, Proc. Comput. Sci., 120 (2017), 126-131.
[5]	J. D. Sitton, Y. Zeinali, B. A. Story, Rapid soil classification using artificial neural networks for use in constructing compressed earth blocks, Constr. Build. Mater., 138 (2017), 214-221.
[6]	D. Valero, D. B. Bung, Artificial Neural Networks and pattern recognition for air-water flow velocity estimation using a single-tip optical fibre probe, J. Hydro. Environ. Res., 19 (2018), 150-159.
[7]	C. C. Lai, K. L. Su, Development of an intelligent mobile robot localization system using Kinect RGB-D mapping and neural network, Comput. Electr. Eng., 67 (2018), 620-628.
[8]	O. Erkaymaz, M. Ozer, M. Perc, Performance of small-world feedforward neural networks for the diagnosis of diabetes, Appl. Math. Comput., 311 (2017), 22-28.
[9]	N. Wan, W. X. Shi, S. S. Fan, S. X. Liu, PSO-FNNbased vertical handoff decision algorithm in heterogeneous wireless networks, Proc. Environ. Sci., 11 (2011), 55-62.
[10]	A. A. Hameed, B. Karlik, M. S. Salman, Back-propagation algorithm with variable adaptive momentum, Knowl. Based Syst., 114 (2016), 79-87.
[11]	D. J. Montana, L. Davis, Training feedforward neural networks using genetic algorithms, Proceedings of the 11th International Joint Conference on Artificial Intelligence, 1989. Available from: https://www.ijcai.org.
[12]	C. Reale, K. Gavin, L. Librić, D. Jurić-Kaćunić, Automatic classification of fine-grained soils using CPT measurements and Artificial Neural Networks, Adv. Eng. Inf., 36 (2018),207-215.
[13]	W. Li, Improving particle swarm optimization based on neighborhood and historical memory for training multi-layer perceptron, Information, 9 (2018), 16.
[14]	S. Gambhir, S. K. Malik, Y. Kumar, PSO-ANN based diagnostic model for the earl detection of dengue disease, New Horiz. Trans. Med., 4 (2017, 1-8.
[15]	S. Z. Mirjalili, S. Saremi, S. M. Mirjalili, Designing evolutionary feedforward neural networks using social spider optimization algorithm, Neural Comput. Appl., 26 (2015),1919-1928.
[16]	E. Uzlu, M. Kankal, A. Akpınar, T. Dede, Estimates of energy consumption in Turkey using neural networks with the teaching-learning-based optimization algorithm, Energy, 75 (2014), 295-303.
[17]	I. Aljarah, H. Faris, S. Mirjalili, Optimizing connection weights in neural networks using the whale optimization algorithm, Soft Comput., 22 (2016), 1-15.
[18]	P. A. Kowalski, S. Łukasik, Training neural networks with Krill Herd algorithm, Neural Process. Lett., 44 (2016), 5-17.
[19]	D. A. Alboaneen, H. Tianfield, Y. Zhang, Glowworm swarm optimization for training multi-layer perceptrons, Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, 2017. Available from: https://dl.acm.org/doi/abs/10.1145/3148055.3148075.
[20]	A. A. Heidari, H Faris, I Aljarah, S Mirjalili, An efficient hybrid multilayer perceptron neural network with grasshopper optimization, Soft. Comput., 23 (2019), 7941-7958.
[21]	E. Valian, S. Mohanna, S. Tavakoli, Improved cuckoo search algorithm for feedforward neural network training, Int. J. Artif. Intell. Appl., 2 (2011), 36-43.
[22]	C. Blum, K. Socha, Training feed-forward neural networks with ant colony optimization: an application to pattern classifcation, Fifth International Conference on Hybrid Intelligent Systems (HIS'05), 2005. Available from: https://ieeexplore.ieee.org/abstract/document/1587754.
[23]	K. Socha, C. Blum, An ant colony optimization algorithm for continuous optimization: Application to feed-forward neural network training, Neural Comput. Appl., 16 (2007), 235-247.
[24]	D. Karaboga, B. Akay, C. Ozturk, Artifcial Bee Colony(ABC) optimization algorithm for training feed-forward neural networks, Proceedings of the International Conference on Modeling Decisions for Artifcial Intelligence (MDAI '07), 2007. Available from: https://link.springer.com/chapter/10.1007/978-3-540-73729-2_30.
[25]	S. Mirjalili, How effective is the Grey Wolf optimizer in training multi-layer perceptions, Appl. Intell., 43 (2015),150-161.
[26]	M Branch, A multi-layer perceptron neural network trained by invasive weed optimization for potato color image segmentation, Trends Appl. Sci. Res., 7 (2012), 445-455.
[27]	D. H. Wolpert, W. G. Macready, No free lunch theorems for optimization, IEEE Trans. Evol. Comput., 1 (1997), 67-82.
[28]	S. Mirjalili, A. Lewis. The whale optimization algorithm, Adv. Eng. Soft., 95 (2016), 51-67.
[29]	M. Abdel-Basset, A. N. Hessin, L. Abdel-Fatah, A comprehensive study of cuckoo-inspired algorithms, Neural Comput. Appl., 29 (2018), 345-361.
[30]	A. A. Heidari, P. Pahlavani, An efficient modified grey wolf optimizer with Lévy flight for optimization tasks, Appl. Soft. Comput., 60 (2017), 115-134.
[31]	S. W. Lin, Z. J. Lee, K. C. Ying, C. Y. Lee, Applying hybrid metaheuristics for capacitated vehicle routing problem, Expert. Syst. Appl., 36 (2009), 1505-1512.
[32]	J. Chen, B. Xin, Z. Peng, L. Dou, J. Zhang, Optimal contraction theorem for exploration-exploitation tradeoff in search and optimization, IEEE Trans. Syst. Man Cybern. Part A Syst. Hum., 39 (2009), 680-691.
[33]	R. Li, S. Hu, Y. Wang, M. Yin, A local search algorithm with tabu strategy and perturbation mechanism for generalized vertex cover problem, Neural Comput. Appl., 28 (2017), 1775-1785.
[34]	N. H. Awad, M. Z. Ali, P. N. Suganthan, R. G. Reynolds, CADE: a hybridization of cultural algorithm and differential evolution for numerical optimization, Inf. Sci., 378 (2017), 215-241.
[35]	M. El-Abd, M. Kamel, A taxonomy of cooperative search algorithms, Int. Workshop Hybrid Metaheuristics, 36 (2005), 32-41.
[36]	V. K. Ojha, A. Abraham, V. Snášel, Metaheuristic design of feedforward neural networks: a review of two decades of research, Eng. Appl. Artif. Int., 60 (2017), 97-116.
[37]	P. Sarkar, N. M. Laskar, S. Nath, K. L. Baishnab, S. Chanda, Offset voltage minimization based circuit sizing of CMOS operational amplifier using whale optimization algorithm, J. Inf. Optim. Sci., 39 (2017), 1-17.
[38]	R. V. Rao, V. J. Savsani, D. P. Vakharia, Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems, Comput. Aided Des., 43 (2011), 303-315.
[39]	W. Spendley, G. R. Hext, F. R. Himsworth, Sequential application of simplex designs in optimisation and evolutionary operation, Technometrics, 4 (1962), 441-461.
[40]	A. A. Heidari, P. Pahlavani, An efficient modified grey wolf optimizer with lexvy flight for optimization tasks, Appl. Soft. Comput., 60 (2017),115-134.
[41]	J. Villanueva, Kolmogorov theorem revisited, J. Differ. Equations, 244 (2008), 2251-2276.
[42]	H. Samet, F. Hashemi, T. Ghanbari, Minimum non detection zone for islanding detection using an optimal Artificial Neural Network algorithm based on PSO, Renewable Sustainable Energy Rev., 52 (2015), 1-18.
[43]	D. A. Alboaneen, H. Tianfield, Y. Zhang, Glowworm swarm optimisation for training multi-layer perceptrons, Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, 2017. Available from: https://dl.acm.org/doi/abs/10.1145/3148055.3148075.
[44]	S. Z. Mirjalili, S. Saremi, S. M. Mirjalili, Designing evolutionary feedforward neural networks using social spider optimization algorithm, Neural Comput. Appli., 26 (2015), 1919-1928.
[45]	I. Aljarah, H. Faris, S. Mirjalili, Optimizing connection weights in neural networks using the whale optimization algorithm, Soft Comput., 22 (2016), 1-15.
[46]	H. Wu, Y. Zhou, Q. Luo, M. A. Basset, Training feedforward neural networks using symbiotic organisms search algorithm, Comput. Intell. Neurosci., 2016 (2016).
[47]	J. Derrac, S. García, D. Molina, F. Herrera, A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms, Swarm Evol. Comput., 1 (2011), 3-18.
[48]	J. Li, Q. Luo, L. Liao, Y. Zhou, Using spotted hyena optimizer for training feedforward neural networks, International Conference on Intelligent Computing, 2018. Available from: https://link.springer.com/chapter/10.1007/978-3-319-95957-3_88.

Reader Comments

Your name:*

Email:*
© 2020 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)