As one of continuous concern all over the world, the problem of water quality may cause diseases and poisoning and even endanger people's lives. Therefore, the prediction of water quality is of great significance to the efficient management of water resources. However, existing prediction algorithms not only require more operation time but also have low accuracy. In recent years, neural networks are widely used to predict water quality, and the computational power of individual neurons has attracted more and more attention. The main content of this research is to use a novel dendritic neuron model (DNM) to predict water quality. In DNM, dendrites combine synapses of different states instead of simple linear weighting, which has a better fitting ability compared with traditional neural networks. In addition, a recent optimization algorithm called AMSGrad (Adaptive Gradient Method) has been introduced to improve the performance of the Adam dendritic neuron model (ADNM). The performance of ADNM is compared with that of traditional neural networks, and the simulation results show that ADNM is better than traditional neural networks in mean square error, root mean square error and other indicators. Furthermore, the stability and accuracy of ADNM are better than those of other conventional models. Based on trained neural networks, policymakers and managers can use the model to predict the water quality. Real-time water quality level at the monitoring site can be presented so that measures can be taken to avoid diseases caused by water quality problems.
Citation: Jing Cao, Dong Zhao, Chenlei Tian, Ting Jin, Fei Song. Adopting improved Adam optimizer to train dendritic neuron model for water quality prediction[J]. Mathematical Biosciences and Engineering, 2023, 20(5): 9489-9510. doi: 10.3934/mbe.2023417
As one of continuous concern all over the world, the problem of water quality may cause diseases and poisoning and even endanger people's lives. Therefore, the prediction of water quality is of great significance to the efficient management of water resources. However, existing prediction algorithms not only require more operation time but also have low accuracy. In recent years, neural networks are widely used to predict water quality, and the computational power of individual neurons has attracted more and more attention. The main content of this research is to use a novel dendritic neuron model (DNM) to predict water quality. In DNM, dendrites combine synapses of different states instead of simple linear weighting, which has a better fitting ability compared with traditional neural networks. In addition, a recent optimization algorithm called AMSGrad (Adaptive Gradient Method) has been introduced to improve the performance of the Adam dendritic neuron model (ADNM). The performance of ADNM is compared with that of traditional neural networks, and the simulation results show that ADNM is better than traditional neural networks in mean square error, root mean square error and other indicators. Furthermore, the stability and accuracy of ADNM are better than those of other conventional models. Based on trained neural networks, policymakers and managers can use the model to predict the water quality. Real-time water quality level at the monitoring site can be presented so that measures can be taken to avoid diseases caused by water quality problems.
| [1] |
T. Ma, N. Zhao, Y. Ni, J. Yi, J. P. Wilson, L. He, et al., China's improving inland surface water quality since 2003, Sci. Adv., 6 (2020), eaau3798. https://doi.org/10.1126/sciadv.aau3798 doi: 10.1126/sciadv.aau3798
|
| [2] | N. Nemerow, Scientific Stream Pollution Analysis, Scripta Book Co., 1974. |
| [3] |
O. Kisi, K. S. Parmar, Application of least square support vector machine and multivariate adaptive regression spline models in long term prediction of river water pollution, J. Hydrol., 534 (2016), 104–112. https://doi.org/10.1016/j.jhydrol.2015.12.014 doi: 10.1016/j.jhydrol.2015.12.014
|
| [4] |
Y. Matsuda, A water pollution prediction system by the finite element method, Adv. Water Resour., 2 (1979), 27–34. https://doi.org/10.1016/0309-1708(79)90004-6 doi: 10.1016/0309-1708(79)90004-6
|
| [5] |
G. Tan, J. Yan, C. Gao, S. Yang, Prediction of water quality time series data based on least squares support vector machine, Procedia Eng., 31 (2012), 1194–1199. https://doi.org/10.1016/j.proeng.2012.01.1162 doi: 10.1016/j.proeng.2012.01.1162
|
| [6] |
H. Chen, L. Xu, W. Ai, B. Lin, Q. Feng, K. Cai, Kernel functions embedded in support vector machine learning models for rapid water pollution assessment via near-infrared spectroscopy, Sci. Total Environ., 714 (2020), 136765. https://doi.org/10.1016/j.scitotenv.2020.136765 doi: 10.1016/j.scitotenv.2020.136765
|
| [7] |
S. Moni, E. Aziz, A. P. A. Majeed, M. Malek, The prediction of blue water footprint at Semambu water treatment plant by means of Artificial Neural Networks (ANN) and Support Vector Machine (SVM) models, Phys. Chem. Earth, 123 (2021), 103052. https://doi.org/10.1016/j.pce.2021.103052 doi: 10.1016/j.pce.2021.103052
|
| [8] | Y. Khan, C. S. See, Predicting and analyzing water quality using machine learning: a comprehensive model, in 2016 IEEE Long Island Systems, Applications and Technology Conference (LISAT), (2021), 1–6. https://doi.org/10.1109/LISAT.2016.7494106 |
| [9] |
M. Azrour, J. Mabrouki, G. Fattah, A. Guezzaz, F. Aziz, Machine learning algorithms for efficient water quality prediction, Model. Earth Syst. Environ., 8 (2022), 2793–2801. https://doi.org/10.2166/wqrj.2022.004 doi: 10.2166/wqrj.2022.004
|
| [10] |
N. Noori, L. Kalin, S. Isik, Water quality prediction using SWAT-ANN coupled approach, J. Hydrol., 590 (2020), 125220. https://doi.org/10.1016/j.jhydrol.2020.125220 doi: 10.1016/j.jhydrol.2020.125220
|
| [11] |
L. Kumar, M. S. Afzal, A. Ahmad, Prediction of water turbidity in a marine environment using machine learning: A case study of Hong Kong, Reg. Stud. Mar. Sci., 52 (2022), 102260. https://doi.org/10.1016/j.rsma.2022.102260 doi: 10.1016/j.rsma.2022.102260
|
| [12] |
L. Li, J. Qiao, G. Yu, L. Wang, H. Y. Li, C. Liao, et al., Interpretable tree-based ensemble model for predicting beach water quality, Water Res., 211 (2022), 118078. https://doi.org/10.1016/j.watres.2022.118078 doi: 10.1016/j.watres.2022.118078
|
| [13] |
M. G. Uddin, S. Nash, M. T. M. Diganta, A. Rahman, A. I. Olbert, Robust machine learning algorithms for predicting coastal water quality index, J. Environ. Manage., 321 (2022), 115923. https://doi.org/10.1016/j.jenvman.2022.115923 doi: 10.1016/j.jenvman.2022.115923
|
| [14] |
W. S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biol., 52 (1990), 99–115. https://doi.org/10.1007/BF02459570 doi: 10.1007/BF02459570
|
| [15] |
F. Rosenblatt, The perceptron: a probabilistic model for information storage and organization in the brain, Psychol. Rev., 65 (1958), 386. https://doi.org/10.1037/h0042519 doi: 10.1037/h0042519
|
| [16] |
D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning representations by back-propagating errors, Nature, 323 (1986), 533–536. https://doi.org/10.1038/323533a0 doi: 10.1038/323533a0
|
| [17] |
Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, 86 (1998), 2278–2324. https://doi.org/10.1109/5.726791 doi: 10.1109/5.726791
|
| [18] |
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, et al., Backpropagation applied to handwritten zip code recognition, Neural Comput., 1 (1989), 541–551. https://doi.org/10.1162/neco.1989.1.4.541 doi: 10.1162/neco.1989.1.4.541
|
| [19] | T. Mikolov, M. Karafiát, L. Burget, J. Cernocky, S. Khudanpur, Recurrent neural network based language model, Interspeech, 2 (2010), 1045–1048. |
| [20] | K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. |
| [21] |
T. Dawood, E. Elwakil, H. M. Novoa, J. F. G. Delgado, Toward urban sustainability and clean potable water: Prediction of water quality via artificial neural networks, J. Cleaner Prod., 291 (2021), 125266. https://doi.org/10.1016/j.jclepro.2020.125266 doi: 10.1016/j.jclepro.2020.125266
|
| [22] |
T. A. Sinshaw, C. Q. Surbeck, H. Yasarer, Y. Najjar, Artificial neural network for prediction of total nitrogen and phosphorus in US lakes, J. Environ. Eng., 145 (2019), 04019032. https://doi.org/10.1061/(ASCE)EE.1943-7870.0001528 doi: 10.1061/(ASCE)EE.1943-7870.0001528
|
| [23] |
M. Hameed, S. S. Sharqi, Z. M. Yaseen, H. A. Afan, A. Hussain, A. Elshafie, Application of artificial intelligence (AI) techniques in water quality index prediction: a case study in tropical region, Malaysia, Neural Comput. Appl., 28 (2017), 893–905. https://doi.org/10.1007/s00521-016-2404-7 doi: 10.1007/s00521-016-2404-7
|
| [24] |
A. Kadam, V. Wagh, A. Muley, B. Umrikar, R. Sankhua, Prediction of water quality index using artificial neural network and multiple linear regression modelling approach in Shivganga River basin, India, Model. Earth Syst. Environ., 5 (2019), 951–962. https://doi.org/10.1007/s40808-019-00581-3 doi: 10.1007/s40808-019-00581-3
|
| [25] |
Y. Zhang, X. Gao, K. Smith, G. Inial, S. Liu, L. B. Conil, et al., Integrating water quality and operation into prediction of water production in drinking water treatment plants by genetic algorithm enhanced artificial neural network, Water Res., 164 (2019), 114888. https://doi.org/10.1016/j.watres.2019.114888 doi: 10.1016/j.watres.2019.114888
|
| [26] |
J. Wu, Z. Wang, A hybrid model for water quality prediction based on an artificial neural network, wavelet transform, and long short-term memory, Water, 14 (2022), 610. https://doi.org/10.3390/w14040610 doi: 10.3390/w14040610
|
| [27] | Y. Wang, J. Zhou, K. Chen, Y. Wang, L. Liu, Water quality prediction method based on LSTM neural network, in 2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), (2017), 1–5. https://doi.org/10.1109/ISKE.2017.8258814 |
| [28] | Q. Ye, X. Yang, C. Chen, J. Wang, River water quality parameters prediction method based on LSTM-RNN model, in 2019 Chinese Control And Decision Conference (CCDC), (2019), 3024–3028. https://doi.org/10.1109/CCDC.2019.8832885 |
| [29] |
J. Bi, Y. Lin, Q. Dong, H. Yuan, M. Zhou, Large-scale water quality prediction with integrated deep neural network, Inf. Sci., 571 (2021), 191–205. https://doi.org/10.1016/j.ins.2021.04.057 doi: 10.1016/j.ins.2021.04.057
|
| [30] | C. Hu, F. Zhao, Improved methods of BP neural network algorithm and its limitation, in 2010 International Forum on Information Technology and Applications, (2010), 11–14. https://doi.org/10.1109/IFITA.2010.324 |
| [31] | T. Venkateswarlu, J. Anmala, Application of random forest model in the prediction of river water quality, in Proceedings of Seventh International Congress on Information and Communication Technology, (2023), 525–535. https://doi.org/10.1016/j.asej.2021.11.004 |
| [32] |
M. Jeung, S. Baek, J. Beom, K. H. Cho, Y. Her, K. Yoon, Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments, J. Hydrol., 575 (2019), 1099–1110. https://doi.org/10.1016/j.jhydrol.2019.05.079 doi: 10.1016/j.jhydrol.2019.05.079
|
| [33] |
H. Lu, X. Ma, Hybrid decision tree-based machine learning models for short-term water quality prediction, Chemosphere, 249 (2020), 126169. https://doi.org/10.1016/j.chemosphere.2020.126169 doi: 10.1016/j.chemosphere.2020.126169
|
| [34] |
S. M. Saghebian, M. T. Sattari, R. Mirabbasi, M. Pal, Ground water quality classification by decision tree method in Ardebil region, Iran, Arabian J. Geosci., 7 (2014), 4767–4777. https://doi.org/10.1007/s12517-013-1042-y doi: 10.1007/s12517-013-1042-y
|
| [35] | Z. Hippe, J. Zamorska, A new approach to application of pattern recognition methods in analytical chemistry. Ⅱ. Prediction of missing values in water pollution grid using modified KNN-method, Chem. Anal., 44 (1999), 597–602. |
| [36] |
J. Park, W. H. Lee, K. T. Kim, C. Y. Park, S. Lee, T. Y. Heo, Interpretation of ensemble learning to predict water quality using explainable artificial intelligence, Sci. Total Environ., 832 (2022), 155070. https://doi.org/10.1016/j.scitotenv.2022.155070 doi: 10.1016/j.scitotenv.2022.155070
|
| [37] |
A. Gidon, T. A. Zolnik, P. Fidzinski, F. Bolduan, A. Papoutsi, P. Poirazi, et al., Dendritic action potentials and computation in human layer 2/3 cortical neurons, Science, 367 (2020), 83–87. https://doi.org/10.1126/science.aax6239 doi: 10.1126/science.aax6239
|
| [38] |
I. S. Jones, K. P. Kording, Might a single neuron solve interesting machine learning problems through successive computations on its dendritic tree?, Neural Comput., 33 (2021), 1554–1571. https://doi.org/10.1162/neco_a_01390 doi: 10.1162/neco_a_01390
|
| [39] |
A. Destexhe, E. Marder, Plasticity in single neuron and circuit computations, Nature, 431 (2004), 789–795. https://doi.org/10.1038/nature03011 doi: 10.1038/nature03011
|
| [40] |
C. Koch, Computation and the single neuron, Nature, 385 (1997), 207–210. https://doi.org/10.1038/385207a0 doi: 10.1038/385207a0
|
| [41] |
B. E. Stein, T. R. Stanford, B. A. Rowland, Development of multisensory integration from the perspective of the individual neuron, Nat. Rev. Neurosci., 15 (2014), 520–535. https://doi.org/10.1038/nrn3742 doi: 10.1038/nrn3742
|
| [42] |
Y. Todo, H. Tamura, K. Yamashita, Z. Tang, Unsupervised learnable neuron model with nonlinear interaction on dendrites, Neural Netw., 60 (2014), 96–103. https://doi.org/10.1016/j.neunet.2014.07.011 doi: 10.1016/j.neunet.2014.07.011
|
| [43] | F. Teng, Y. Todo, Dendritic neuron model and its capability of approximation, in 2019 6th International Conference on Systems and Informatics (ICSAI), (2019), 542–546. https://doi.org/10.1109/ICSAI48974.2019.9010147 |
| [44] | J. He, J. Wu, G. Yuan, Y. Todo, Dendritic branches of dnm help to improve approximation accuracy, in 2019 6th International Conference on Systems and Informatics (ICSAI), (2019), 533–541. https://doi.org/10.1109/ICSAI48974.2019.9010196 |
| [45] |
Z. Sha, L. Hu, Y. Todo, J. Ji, S. Gao, Z. Tang, A breast cancer classifier using a neuron model with dendritic nonlinearity, IEICE Trans. Commun., 98 (2015), 1365–1376. https://doi.org/10.1587/transinf.2014EDP7418 doi: 10.1587/transinf.2014EDP7418
|
| [46] |
T. Jiang, S. Gao, D. Wang, J. Ji, Y. Todo, Z. Tang, A neuron model with synaptic nonlinearities in a dendritic tree for liver disorder, IEEJ Trans. Electr. Electron. Eng., 12 (2017), 105–115. https://doi.org/10.1002/tee.22350 doi: 10.1002/tee.22350
|
| [47] |
Y. Tang, J. Ji, S. Gao, H. Dai, Y. Yu, Y. Todo, A pruning neural network model in credit classification analysis, Comput. Intell. Neurosci., 15 (2014), 520–535. https://doi.org/10.1155/2018/9390410 doi: 10.1155/2018/9390410
|
| [48] |
Z. Song, C. Tang, J. Ji, Y. Todo, Z. Tang, A simple dendritic neural network model-based approach for daily pm2.5 concentration prediction, Electronics, 10 (2021), 373. https://doi.org/10.3390/electronics10040373 doi: 10.3390/electronics10040373
|
| [49] |
Z. Song, Y. Tang, J. Ji, Y. Todo, Evaluating a dendritic neuron model for wind speed forecasting, Knowl. Based Syst., 201 (2020), 106052. https://doi.org/10.1016/j.knosys.2020.106052 doi: 10.1016/j.knosys.2020.106052
|
| [50] |
T. Zhou, S. Gao, J. Wang, C. Chu, Y. Todo, Z. Tang, Financial time series prediction using a dendritic neuron model, Knowl. Based Syst., 105 (2016), 214–224. https://doi.org/10.1016/j.knosys.2016.05.031 doi: 10.1016/j.knosys.2016.05.031
|
| [51] |
W. Chen, J. Sun, S. Gao, J. J. Cheng, J. Wang, Y. Todo, Using a single dendritic neuron to forecast tourist arrivals to japan, IEICE Trans. Inf. Syst., 100 (2017), 190–202. https://doi.org/10.1587/transinf.2016EDP7152 doi: 10.1587/transinf.2016EDP7152
|
| [52] | S. J. Reddi, S. Kale, S. Kumar, On the convergence of adam and beyond, preprint, arXiv: 1904.09237. https://doi.org/10.48550/arXiv.1904.09237 |
| [53] | D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, preprint, arXiv: 1412.6980. https://doi.org/10.48550/arXiv.1412.6980 |
| [54] |
J. Ji, M. Dong, Q. Lin, K. C. Tan, Noninvasive cuffless blood pressure estimation with dendritic neural regression, IEEE Trans. Cybern., 2022 (2022). https://doi.org/10.1109/TCYB.2022.3141380 doi: 10.1109/TCYB.2022.3141380
|
| [55] |
J. F. Khaw, B. Lim, L. E. Lim, Optimal design of neural networks using the taguchi method, Neurocomputing, 7 (1995), 225–245. https://doi.org/10.1016/0925-2312(94)00013-I doi: 10.1016/0925-2312(94)00013-I
|
| [56] | D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning internal representations by error propagation, in California Univ San Diego La Jolla Inst for Cognitive Science, 1985. |
| [57] |
J. H. Friedman, , Greedy function approximation: a gradient boosting machine, Ann. Stat., 2001 (2001), 1189–1232. https://doi.org/10.1214/AOS/1013203451 doi: 10.1214/AOS/1013203451
|
| [58] | T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, in Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, (2016), 785–794. https://doi.org/10.1145/2939672.2939785 |
| [59] |
G. B. Huang, Q. Y. Zhu, C. K. Siew, Extreme learning machine: theory and applications, Neurocomputing, 70 (2006), 489–501. https://doi.org/10.1016/j.neucom.2005.12.126 doi: 10.1016/j.neucom.2005.12.126
|
| [60] |
D. W. Zimmerman, B. D. Zumbo, Relative power of the Wilcoxon test, the Friedman test, and repeated-measures ANOVA on ranks, J. Exp. Educ., 62 (1993), 75–86. https://doi.org/10.1080/00220973.1993.9943832 doi: 10.1080/00220973.1993.9943832
|
Figures(9) / Tables(6)
Jing Cao, Dong Zhao, Chenlei Tian, Ting Jin, Fei Song. Adopting improved Adam optimizer to train dendritic neuron model for water quality prediction[J]. Mathematical Biosciences and Engineering, 2023, 20(5): 9489-9510. doi: 10.3934/mbe.2023417
DownLoad: