Perceptual loss function for generating high-resolution climate data

Yang Wang; Hassan A. Karimi; Yang Wang; Hassan A. Karimi

doi:10.3934/aci.2022009

Applied Computing and Intelligence

2022, Volume 2, Issue 2: 152-172. doi: 10.3934/aci.2022009

Previous Article Next Article

Research article

Perceptual loss function for generating high-resolution climate data

Yang Wang ^,,
Hassan A. Karimi

Geoinformatics Laboratory, School of Computing and Information, University of Pittsburgh, 135 N Bellefield Ave, Pittsburgh, PA 15213, USA

Academic Editor: Pasi Fränti

Received: 09 August 2022 Revised: 07 October 2022 Accepted: 07 October 2022 Published: 13 October 2022

When planning the development of future energy resources, electrical infrastructure, transportation networks, agriculture, and many other societally significant systems, policy makers require accurate and high-resolution data reflecting different climate scenarios. There is widely documented evidence that perceptual loss can be used to generate perceptually realistic results when mapping low-resolution inputs to high-resolution outputs, but its application is limited to images at present. In this paper, we study the perceptual loss when increasing the resolution of raw precipitation data by ×4 and ×8 under training modes of CNN and GAN. We examine the difference in the perceptual loss calculated by using different layers of feature maps and demonstrate how low- and mid-level feature maps can yield comparable results to pixel-wise loss. In particular, from both qualitative and quantitative points of view, Conv2_1 and Conv3_1 are the best compromises between obtaining detailed information and maintaining the overall error in our case. We propose a new approach to benefit from perceptual loss while considering the characteristics of climate data. We show that in comparison to calculating perceptual loss directly for the entire sample, our proposed approach can obtain detailed information of extreme events regions while reducing error.
- climate downscaling,
- perceptual loss,
- deep learning,
- GAN
Citation: Yang Wang, Hassan A. Karimi. Perceptual loss function for generating high-resolution climate data[J]. Applied Computing and Intelligence, 2022, 2(2): 152-172. doi: 10.3934/aci.2022009

Related Papers:

Abstract

When planning the development of future energy resources, electrical infrastructure, transportation networks, agriculture, and many other societally significant systems, policy makers require accurate and high-resolution data reflecting different climate scenarios. There is widely documented evidence that perceptual loss can be used to generate perceptually realistic results when mapping low-resolution inputs to high-resolution outputs, but its application is limited to images at present. In this paper, we study the perceptual loss when increasing the resolution of raw precipitation data by ×4 and ×8 under training modes of CNN and GAN. We examine the difference in the perceptual loss calculated by using different layers of feature maps and demonstrate how low- and mid-level feature maps can yield comparable results to pixel-wise loss. In particular, from both qualitative and quantitative points of view, Conv2_1 and Conv3_1 are the best compromises between obtaining detailed information and maintaining the overall error in our case. We propose a new approach to benefit from perceptual loss while considering the characteristics of climate data. We show that in comparison to calculating perceptual loss directly for the entire sample, our proposed approach can obtain detailed information of extreme events regions while reducing error.

References

[1]	F. K. Muthoni, V. O. Odongo, J. Ochieng, E. M. Mugalavai, S. K. Mourice, I. Hoesche-zeledon, et al., Long-term spatial-temporal trends and variability of rainfall over Eastern and Southern Africa. Theor. Appl. Climatol., 137 (2019), 1869–1882. https://doi.org/10.1007/s00704-018-2712-1 doi: 10.1007/s00704-018-2712-1
[2]	C. C. Wei, Simulation of operational typhoon rainfall nowcasting using radar reflectivity combined with meteorological data, J. Geophys. Res.-Atmos., 119 (2014), 6578–6595. https://doi.org/10.1002/2014JD021488 doi: 10.1002/2014JD021488
[3]	C. P. Castillo, F. B. e Silva, C. Lavalle, An assessment of the regional potential for solar power generation in EU-28, Energy Policy, 88 (2016), 86–99. https://doi.org/10.1016/j.enpol.2015.10.004 doi: 10.1016/j.enpol.2015.10.004
[4]	A. Voldoire, E. Sanchez-Gomez, D. Salas y Mélia, B. Decharme, C. Cassou, S. Sénési, et al., The CNRM-CM5.1 global climate model: Description and basic evaluation, Clim. Dyn., 40 (2013), 2091–2121. https://doi.org/10.1007/s00382-011-1259-y doi: 10.1007/s00382-011-1259-y
[5]	N. Hofstra, M. Haylock, M. New, P. Jones, C. Frei, Comparison of six methods for the interpolation of daily, European climate data, J. Geophys. Res. Atmos., 113 (2008). https://doi.org/10.1029/2008JD010100 doi: 10.1029/2008JD010100
[6]	C. Dong, C. C. Loy, K. He, X. Tang, Image Super-Resolution Using Deep Convolutional Networks, IEEE T. Pattern Anal. Mach. Intell., 38 (2015), 295–307. https://doi.org/10.1109/TPAMI.2015.2439281 doi: 10.1109/TPAMI.2015.2439281
[7]	C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, et al., Photo-realistic single image super-resolution using a generative adversarial network, Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, 4681–4690. https://doi.org/10.1109/CVPR.2017.19
[8]	K. Stengel, A. Glaws, D. Hettinger, R. N. King, Adversarial super-resolution of climatological wind and solar data, Proc. Natl. Acad. Sci. U. S. A., 117 (2020), 16805–16815. https://doi.org/10.1073/pnas.1918964117 doi: 10.1073/pnas.1918964117
[9]	T. Vandal, E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, A. R. Ganguly, Generating high resolution climate change projections through single image super-resolution: An abridged version, IJCAI Int. Jt. Conf. Artif. Intell., (2018) 5389–5393. https://doi.org/10.24963/ijcai.2018/759 doi: 10.24963/ijcai.2018/759
[10]	E. R. Rodrigues, I. Oliveira, R. Cunha, M. Netto, DeepDownscale: A deep learning strategy for high-resolution weather forecast, Proc. - IEEE 14th Int. Conf. eScience, e-Science, (2018), 415–422. https://doi.org/10.1109/eScience.2018.00130 doi: 10.1109/eScience.2018.00130
[11]	Y. Jo, S. Yang, S. J. Kim, Investigating loss functions for extreme super-resolution, IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., (2020), 424–425.
[12]	J. Johnson, A. Alahi, L. Fei-Fei, Perceptual losses for real-time style transfer and super-resolution, Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 9906 (2016), 694–711. https://doi.org/10.1007/978-3-319-46475-6_43 doi: 10.1007/978-3-319-46475-6_43
[13]	M. S. Sajjadi, B. Scholkopf, M. Hirsch, EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis, Proc. IEEE Int. Conf. Comput. Vis., (2017), 4491–4500. https://doi.org/10.1109/ICCV.2017.481 doi: 10.1109/ICCV.2017.481
[14]	K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., (2015), 1–14.
[15]	A. Mahendran, A. Vedaldi, Visualizing Deep Convolutional Neural Networks Using Natural Pre-images, Int. J. Comput. Vis., 120 (2016), 233–255. https://doi.org/10.1007/s11263-016-0911-8 doi: 10.1007/s11263-016-0911-8
[16]	W. Yu, K. Yang, Y. Bai, H. Yao, Y. Rui, Visualizing and Comparing Convolutional Neural Networks, 2014.
[17]	Y. Ma, K. A. Lee, V. Hautamaki, H. Li, PL-EESR: Perceptual Loss Based End-to-End Robust Speaker Representation Extraction, 2021 IEEE Autom. Speech Recognit. Underst. Work. ASRU, (2021), 106–113.
[18]	E. Shelhamer, J. Long, T. Darrell, Fully Convolutional Networks for Semantic Segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2015), 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965 doi: 10.1109/CVPR.2015.7298965
[19]	Y. Liu, H. Chen, Y. Chen, W. Yin, C. Shen, Generic Perceptual Loss for Modeling Structured Output Dependencies, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., (2021), 5420–5428. https://doi.org/10.1109/CVPR46437.2021.00538 doi: 10.1109/CVPR46437.2021.00538
[20]	R. Huth, Statistical downscaling of daily temperature in central Europe, J. Clim., 15 (2002), 1731–1742. https://doi.org/10.1175/1520-0442(2002)015<1731:SDODTI>2.0.CO;2 doi: 10.1175/1520-0442(2002)015<1731:SDODTI>2.0.CO;2
[21]	S. T. Chen, P. S. Yu, Y. H. Tang, Statistical downscaling of daily precipitation using support vector machines and multivariate analysis, J. Hydrol., 385 (2010), 13–22. https://doi.org/10.1016/j.jhydrol.2010.01.021 doi: 10.1016/j.jhydrol.2010.01.021
[22]	C. Hutengs, M. Vohland, Downscaling land surface temperatures at regional scales with random forest regression, Remote Sens. Environ., 178 (2016), 127–141. https://doi.org/10.1016/j.rse.2016.03.006 doi: 10.1016/j.rse.2016.03.006
[23]	T. Vandal, E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, A. R. Ganguly, DeepSD: Generating high resolution climate change projections through single image super-resolution, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., (2017), 1663–1672. https://doi.org/10.1145/3097983.3098004 doi: 10.1145/3097983.3098004
[24]	J. Cheng, Q. Kuang, C. Shen, J. Liu, X. Tan, W. Liu, ResLap: Generating High-Resolution Climate Prediction through Image Super-Resolution, IEEE Access, 8 (2020), 39623–39634. https://doi.org/10.1109/ACCESS.2020.2974785 doi: 10.1109/ACCESS.2020.2974785
[25]	C. Besombes, O. Pannekoucke, C. Lapeyre, B. Sanderson, O. Thual, Producing realistic climate data with generative adversarial networks, Nonlinear Proc. Geoph., 28 (2021), 347–370. https://doi.org/10.5194/npg-28-347-2021 doi: 10.5194/npg-28-347-2021
[26]	J. Bruna, P. Sprechmann, Y. LeCun, Super-resolution with deep convolutional sufficient statistics, 4th Int. Conf. Learn. Represent. ICLR 2016 - Conf. Track Proc., (2016), 1–17.
[27]	X. Xu, M. Xie, P. Miao, W. Qu, W. Xiao, H. Zhang, et al., Perceptual-Aware Sketch Simplification Based on Integrated VGG Layers, IEEE T. Vis. Comput. Gr., 27 (2019), 178–189. https://doi.org/10.1109/TVCG.2019.2930512 doi: 10.1109/TVCG.2019.2930512
[28]	A. Mahendran, A. Vedaldi, Understanding deep image representations by inverting them, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., (2015), 5188–5196. https://doi.org/10.1109/CVPR.2015.7299155 doi: 10.1109/CVPR.2015.7299155
[29]	Q. Yang, P. Yan, Y. Zhang, H. Yu, Y. Shi, X. Mou, et al., Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss, IEEE T. Med. Imaging, 37 (2018), 1348–1357. https://doi.org/10.1109/TMI.2018.2827462 doi: 10.1109/TMI.2018.2827462
[30]	P. Sangkloy, J. Lu, C. Fang, F. Yu, J. Hays, Scribbler: Controlling deep image synthesis with sketch and color, Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, (2017), 6836–6845. https://doi.org/10.1109/CVPR.2017.723 doi: 10.1109/CVPR.2017.723
[31]	M. S. Rad, B. Bozorgtabar, U. V. Marti, M. Basler, H. K. Ekenel, J. P. Thiran, SROBB: Targeted perceptual loss for single image super-resolution, Proc. IEEE Int. Conf. Comput. Vis., (2019), 2710–2719.
[32]	Y. Zhang, W. Li, W. Gong, Z. Wang, J. Sun, An improved boundary-aware perceptual loss for building extraction from VHR images, Remote Sens., 12 (2020), 1195. https://doi.org/10.3390/rs12071195 doi: 10.3390/rs12071195
[33]	J. Chi, J. Bae, Y. J. Kwon, Two-stream convolutional long-and short-term memory model using perceptual loss for sequence-to-sequence arctic sea ice prediction, Remote Sens., 13 (2021), 3413. https://doi.org/10.3390/rs13173413 doi: 10.3390/rs13173413
[34]	G. Skofronick-Jackson, W. A. Petersen, W. Berg, C. Kidd, E. F. Stocker, D. B. Kirschbaum, et al., The global precipitation measurement (GPM) mission for science and Society, B. Am. Meteorol. Soc., 98 (2017), 1679–1695.
[35]	V. Levizzanni, P. Bauer, F. J. Turk, Measuring Precipitation from Space: EURAINSAT and the Future, vol. 28, 2007. https://doi.org/10.1007/978-1-4020-5835-6
[36]	J. P. Dunne, L. W. Horowitz, A. J. Adcroft, P. Ginoux, I. M. Held, J. G. John, et al., The GFDL Earth System Model Version 4.1 (GFDL-ESM 4.1): Overall Coupled Model Description and Simulation Characteristics, J. Adv. Model. Earth Syst., 12 (2020), e2019MS002015. https://doi.org/10.1029/2019MS002015 doi: 10.1029/2019MS002015

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)