A time series image prediction method combining a CNN and LSTM and its application in typhoon track prediction

Peng Lu; Ao Sun; Mingyu Xu; Zhenhua Wang; Zongsheng Zheng; Yating Xie; Wenjuan Wang; Peng Lu; Ao Sun; Mingyu Xu; Zhenhua Wang; Zongsheng Zheng; Yating Xie; Wenjuan Wang

doi:10.3934/mbe.2022571

Mathematical Biosciences and Engineering

2022, Volume 19, Issue 12: 12260-12278. doi: 10.3934/mbe.2022571

Previous Article Next Article

Research article Special Issues

A time series image prediction method combining a CNN and LSTM and its application in typhoon track prediction

College of Information Technology, Shanghai Ocean University, Shanghai 201306, China

Academic Editor: Runzhang Xu

Received: 16 May 2022 Revised: 24 July 2022 Accepted: 11 August 2022 Published: 22 August 2022

Typhoon forecasting has always been a vital function of the meteorological department. Accurate typhoon forecasts can provide a priori information for the relevant meteorological departments and help make more scientific decisions to reduce the losses caused by typhoons. However, current mainstream typhoon forecast methods are very challenging and expensive due to the complexity of typhoon motion and the scarcity of ocean observation stations. In this paper, we propose a typhoon track prediction model, DeepTyphoon, which integrates convolutional neural networks and long short-term memory (LSTM). To establish the relationship between the satellite image and the typhoon center, we mark the typhoon center on the satellite image. Then, we use hybrid dilated convolution to extract the cloud features of the typhoon from satellite images and use LSTM to predict these features. Finally, we detect the location of the typhoon according to the predictive markers in the output image. Experiments are conducted using 13, 400 satellite images of time series of the Northwest Pacific from 1980 to 2020 and 8420 satellite images of time series of the Southwest Pacific released by the Japan Meteorological Agency. From the experimentation, the mean average error of the 6-hour typhoon prediction result is 64.17 km, which shows that the DeepTyphoon prediction model significantly outperforms existing deep learning approaches. It achieves successful typhoon track prediction based on satellite images.
- typhoon track prediction,
- CNN,
- LSTM,
- satellite image
Citation: Peng Lu, Ao Sun, Mingyu Xu, Zhenhua Wang, Zongsheng Zheng, Yating Xie, Wenjuan Wang. A time series image prediction method combining a CNN and LSTM and its application in typhoon track prediction[J]. Mathematical Biosciences and Engineering, 2022, 19(12): 12260-12278. doi: 10.3934/mbe.2022571

Related Papers:

Abstract

Typhoon forecasting has always been a vital function of the meteorological department. Accurate typhoon forecasts can provide a priori information for the relevant meteorological departments and help make more scientific decisions to reduce the losses caused by typhoons. However, current mainstream typhoon forecast methods are very challenging and expensive due to the complexity of typhoon motion and the scarcity of ocean observation stations. In this paper, we propose a typhoon track prediction model, DeepTyphoon, which integrates convolutional neural networks and long short-term memory (LSTM). To establish the relationship between the satellite image and the typhoon center, we mark the typhoon center on the satellite image. Then, we use hybrid dilated convolution to extract the cloud features of the typhoon from satellite images and use LSTM to predict these features. Finally, we detect the location of the typhoon according to the predictive markers in the output image. Experiments are conducted using 13, 400 satellite images of time series of the Northwest Pacific from 1980 to 2020 and 8420 satellite images of time series of the Southwest Pacific released by the Japan Meteorological Agency. From the experimentation, the mean average error of the 6-hour typhoon prediction result is 64.17 km, which shows that the DeepTyphoon prediction model significantly outperforms existing deep learning approaches. It achieves successful typhoon track prediction based on satellite images.

References

[1]	L. S. Chen, Y. H. Ding, Introduction to Typhoons in The Western Pacific, Science Press, 1979.
[2]	Y. Sun, Z. Zhong, T. Li, L. Yi, Y. J. Hu, H. C. Wan, et al., Impact of ocean warming on tropical cyclone size and its destructiveness, Sci. Rep., 7 (2017), 1–10. https://doi.org/10.1038/s41598-017-08533-6 doi: 10.1038/s41598-016-0028-x
[3]	J. P. Kossin, A global slowdown of tropical–cyclone translation speed, Nature, 558 (2018), 104–107. https://doi.org/10.1038/s41586-018-0158-3 doi: 10.1038/s41586-018-0158-3
[4]	H. Kim, U. Yun, B. Vo, J. C. W. Lin, W. Pedrycz, Periodicity-oriented data analytics on time-series data for intelligence system, IEEE Syst. J., 15 (2021), 4958–4969. https://doi.org/10.1109/JSYST.2020.3022640 doi: 10.1109/JSYST.2020.3022640
[5]	T. T. Tang, Q. L. Li, G. X. Li, Y. L. Peng, Research on statistical model of typhoon intensity prediction based on meteorological big data, Integr. Technol., 2 (2016), 73–84. https://doi.org/10.12146/j.issn.2095-3135.201602006 doi: 10.12146/j.issn.2095-3135.201602006
[6]	T. Song, Y. Li, F. Meng, P. F. Xie, D. Y. Xu, A novel deep learning model by BiGRU with attention mechanism for tropical cyclone track prediction in the northwest pacific, J. Appl. Meteorol. Climatol., 61 (2022), 3–12. https://doi.org/10.1175/JAMC-D-20-0291.1 doi: 10.1175/JAMC-D-20-0291.1
[7]	J. Lian, P. P. Dong, Y. P. Zhang, J. G. Pan, K. H. Liu, A novel data–driven tropical cyclone track prediction model based on CNN and GRU with multi-dimensional feature selection, IEEE Access, 8 (2020), 97114–97128. https://doi.org/10.1109/ACCESS.2020.2992083 doi: 10.1109/ACCESS.2020.2992083
[8]	S. Gao, P. Zhao, B. Pan, Y. R. Li, M. Zhou, J. L. Xu, et al., A nowcasting model for the prediction of typhoon tracks based on a long short term memory neural network, Acta Oceanol. Sin., 37 (2018), 8–12. https://doi.org/10.1007/s13131-018-1219-z doi: 10.1007/s13131-018-1219-z
[9]	R. S. T. Lee, J. N. K. Liu, Tropical cyclone identification and tracking system using integrated neural oscillatory elastic graph matching and hybrid RBF network track mining techniques, IEEE Trans. Neural Network, 11 (2000), 680–689. https://doi.org/10.1109/72.846739 doi: 10.1109/72.846739
[10]	R. Kovordányi, C. Roy, Cyclone track forecasting based on satellite images using artificial neural networks, ISPRS J. Photogramm., 64 (2009), 513–521. https://doi.org/10.1016/j.isprsjprs.2009.03.002 doi: 10.1016/j.isprsjprs.2009.03.002
[11]	M. Rüttgers, S. Lee, S. Jeon, D. You, Prediction of a typhoon track using a generative adversarial network and satellite images, Sci. Rep., 9 (2019), 1–15. https://doi.org/10.1038/s41598-019-42339-y doi: 10.1038/s41598-018-37186-2
[12]	I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., Generative adversarial networks, Commun. ACM, 63 (2020), 139–144. https://doi.org/10.1145/3422622 doi: 10.1145/3422622
[13]	K. Q. Liu, J. Y. Yuan, W. Z. Chen, Y. P. Chen, Verification of typhoon storm surge forecast model based on adcirc hydrodynamic model, Water Resour. Hydropower Express, 40 (2019), 27–34. https://doi.org/10.15974/j.cnki.slsdkb.2019.04.005 doi: 10.15974/j.cnki.slsdkb.2019.04.005
[14]	L. G. Wu, Z. P. Ni, J. J. Duan, H. J. Zong, Sudden tropical cyclone track changes over the western north pacific: A composite study, Mon. Weather Rev., 141 (2013), 2597–2610. https://doi.org/10.1175/MWR-D-12-00224.1 doi: 10.1175/MWR-D-12-00224.1
[15]	Z. Xu, J. Du, J. J. Wang, C. X. Jiang, Y. Ren, Satellite image prediction relying on GAN and LSTM neural networks, in IEEE International Conference on Communications, (2019), 1–6. https://doi.org/10.1109/ICC.2019.8761462
[16]	S. Kim, J. S. Kang, M. H. Lee, S. Song, Deeptc: Convlstm network for trajectory prediction of tropical cyclone using spatiotemporal atmospheric simulation data, in Conference and Workshop on Neural Information Processing Systems (NIPS), (2018). https://openreview.net/pdf?id=HJlCVoPAF7
[17]	C. Tan, Tclnet: Learning to locate typhoon center using deep neural network, in International Geoscience and Remote Sensing Symposium (IGARSS), (2021), 4600–4603. https://doi.org/10.1109/IGARSS47720.2021.9554524
[18]	R. Pradhan, R. S. Aygun, M. Maskey, R. Ramachandran, D. J. Cecil, Tropical cyclone intensity estimation using a deep convolutional neural network, IEEE Trans. Image Process., 27 (2018), 692–702. https://doi.org/10.1109/TIP.2017.2766358 doi: 10.1109/TIP.2017.2766358
[19]	S. Woo, J. Park, J. Y. Lee, I. S. Kweon, CBAM: Convolutional block attention module, in Proceedings of the European conference on computer vision (ECCV), (2018), 3–19. https://doi.org/10.1007/978-3-030-01234-2-1
[20]	S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., 9 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 doi: 10.1162/neco.1997.9.8.1735
[21]	A. Graves, Supervised sequence labelling with recurrent neural networks, in Studies in Computational Intelligence, 385 (2008). https://doi.org/10.1007/978-3-642-24797-2
[22]	F. Yu, V. Koltun, T. Funkhouser, Dilated residual networks, in Proceedings-30th IEEE Conference on Computer Vision and Pattern Recognition, (2017), 636–644. https://doi.org/10.1109/CVPR.2017.75
[23]	P. Q. Wang, P. F. Chen, Y. Yuan, D. Liu, Z. H. Huang, X. D. Hou, et al., Understanding Convolution for Semantic Segmentation, in Proceedings-2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, (2018), 1451–1460. https://doi.org/10.1109/WACV.2018.00163
[24]	J. B. Yu, The Research on the Recognition of the Classification of Cloud, Typhoon Segmentation and Locating of the Typhoon's Centre Based on the Meteorological Satellite, Master's thesis, Wuhan University of Technology, 2008. https://doi.org/10.7666/d.y1365665
[25]	H. Mahmoud, N. Akkari, Shortest path calculation: A comparative study for location-based recommender system, in Proceedings-2016 World Symposium on Computer Applications and Research, (2016), 1–5. https://doi.org/10.1109/WSCAR.2016.16
[26]	S. Giffard-Roisin, M. Yang, G. Charpiat, C. K. Bonfanti, B. Kégland, C. Monteleoni, Tropical cyclone track forecasting using fused deep learning from aligned reanalysis data, Front. Big Data, 3 (2020), 1. https://doi.org/10.3389/fdata.2020.00001 doi: 10.3389/fdata.2020.00001
[27]	S. Alemany, J. Beltran, A. Perez, S. Ganzfried, Predicting hurricane trajectories using a recurrent neural network, in Proceedings of the AAAI Conference on Artificial Intelligence, (2019), 468–475. https://doi.org/10.1609/aaai.v33i01.3301468
[28]	J. Lian, P. P. Dong, Y. P. Zhang, J. G. Pan, A novel deep learning approach for tropical cyclone track prediction based on auto–encoder and gated recurrent unit networks, Appl. Sci., 10 (2020), 3965. https://doi.org/10.3390/app10113965 doi: 10.3390/app10113965
[29]	S. F. Tekin, F. Ilhan, Cs559 project report forecasting of tropical cyclone trajectories with deep learning, preprint, arXiv: 1910.10566v2. https://doi.org/10.48550/arXiv.1910.10566
[30]	W. Qin, J. Tang, S. Y. Lao, Deepfr: A trajectory prediction model based on deep feature representation, Inf. Sci., 604 (2022), 226–248. https://doi.org/10.1016/j.ins.2022.05.019 doi: 10.1016/j.ins.2022.05.019
[31]	W. Qin, J. Tang, C. Lu, S. Lao, A typhoon trajectory prediction model based on multimodal and multitask learning, Appl. Soft Comput., 122 (2022), 108804. https://doi.org/10.1016/j.asoc.2022.108804 doi: 10.1016/j.asoc.2022.108804

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)