Robust error estimates of PINN in one-dimensional boundary value problems for linear elliptic equations

Jihahm Yoo; Haesung Lee; Jihahm Yoo; Haesung Lee

doi:10.3934/math.20241314

AIMS Mathematics

2024, Volume 9, Issue 10: 27000-27027. doi: 10.3934/math.20241314

Previous Article Next Article

Research article Special Issues

Robust error estimates of PINN in one-dimensional boundary value problems for linear elliptic equations

Jihahm Yoo ¹,
Haesung Lee ^{2
,
,}

1.
Korea Science Academy of KAIST, Busan 47162, Republic of Korea
2.
Department of Mathematics and Big Data Science, Kumoh National Institute of Technology, Gumi, Gyeongsangbuk-do 39177, Republic of Korea

Received: 19 July 2024 Revised: 25 August 2024 Accepted: 04 September 2024 Published: 18 September 2024
MSC : Primary: 34B05, 35A15; Secondary: 68T07, 65L10

In this paper, we study physics-informed neural networks (PINN) to approximate solutions to one-dimensional boundary value problems for linear elliptic equations and establish robust error estimates of PINN regardless of the quantities of the coefficients. In particular, we rigorously demonstrate the existence and uniqueness of solutions using the Sobolev space theory based on a variational approach. Deriving $ L^2 $-contraction estimates, we show that the error, defined as the mean square of the differences between the true solution and our trial function at the sample points, is dominated by the training loss. Furthermore, we show that as the quantities of the coefficients for the differential equation increase, the error-to-loss ratio rapidly decreases. Our theoretical and experimental results confirm the robustness of the error regardless of the quantities of the coefficients.
- Sobolev spaces,
- boundary value problems,
- existence and uniqueness,
- physics-informed neural networks (PINN),
- $ L^2 $-contraction estimates,
- error estimates
Citation: Jihahm Yoo, Haesung Lee. Robust error estimates of PINN in one-dimensional boundary value problems for linear elliptic equations[J]. AIMS Mathematics, 2024, 9(10): 27000-27027. doi: 10.3934/math.20241314

Related Papers:

Abstract

In this paper, we study physics-informed neural networks (PINN) to approximate solutions to one-dimensional boundary value problems for linear elliptic equations and establish robust error estimates of PINN regardless of the quantities of the coefficients. In particular, we rigorously demonstrate the existence and uniqueness of solutions using the Sobolev space theory based on a variational approach. Deriving $ L^2 $-contraction estimates, we show that the error, defined as the mean square of the differences between the true solution and our trial function at the sample points, is dominated by the training loss. Furthermore, we show that as the quantities of the coefficients for the differential equation increase, the error-to-loss ratio rapidly decreases. Our theoretical and experimental results confirm the robustness of the error regardless of the quantities of the coefficients.

References

[1]	A. Biswas, J. Tian, S. Ulusoy, Error estimates for deep learning methods in fluid dynamics, Numer. Math., 151 (2022), 753–777. https://dx.doi.org/10.1007/s00211-022-01294-z doi: 10.1007/s00211-022-01294-z
[2]	H. Brezis, Functional analysis, Sobolev spaces and partial differential equations, New York: Springer, 2011. https://dx.doi.org/10.1007/978-0-387-70914-7
[3]	R. T. Q. Chen, Y. Rubanova, J. Bettencourt, D. K. Duvenaud, Neural ordinary differential equations, Proceedings of the 32nd International Conference on Neural Information Processing Systems, Canada: Montréal, 2018, 6572–6583.
[4]	G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals Syst., 2 (1989), 303–314. https://dx.doi.org/10.1007/BF02551274 doi: 10.1007/BF02551274
[5]	T. De Ryck, S. Mishra, Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs, Adv. Comput. Math., 48 (2022), 1–40. https://dx.doi.org/10.1007/s10444-022-09985-9 doi: 10.1007/s10444-022-09985-9
[6]	T. De Ryck, A. D. Jagtap, S. Mishra, Error estimates for physics-informed neural networks approximating the Navier-Stokes equations, IMA J. Numer. Anal., 44 (2024), 83–119. https://dx.doi.org/10.1093/imanum/drac085 doi: 10.1093/imanum/drac085
[7]	M. W. M. G. Dissanayake, N. Phan-Thien, Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Eng., 10 (1994), 195–201. https://dx.doi.org/10.1002/cnm.1640100303 doi: 10.1002/cnm.1640100303
[8]	P. A. Farrell, A. F. Hegarty, J. J. H. Miller, E. O'Riordan, G. I. Shishkin, Singularly perturbed convection-diffusion problems with boundary and weak interior layers, J. Comput. Appl. Math., 166 (2004), 133–151. https://dx.doi.org/10.1016/j.cam.2003.09.033 doi: 10.1016/j.cam.2003.09.033
[9]	K. Hornik, M. Stinchcombe, H. White, Multilayer feedforward networks are universal approximators, Neural Networks, 2 (1989), 359–366. https://dx.doi.org/10.1016/0893-6080(89)90020-8 doi: 10.1016/0893-6080(89)90020-8
[10]	Y. Hong, C. Y, Jung, J. Laminie, Singularly perturbed reaction-diffusion equations in a circle with numerical applications, Int. J. Comput. Math., 90 (2013), 2308–2325. https://dx.doi.org/10.1080/00207160.2013.772987 doi: 10.1080/00207160.2013.772987
[11]	Y. Hong, S. Ko, J. Lee, Error analysis for finite element operator learning methods for solving parametric second-order elliptic PDEs, arXiv, 2024. https://dx.doi.org/10.48550/arXiv.2404.17868
[12]	I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Networks, 9 (1998), 987–1000. https://dx.doi.org/10.1109/72.712178 doi: 10.1109/72.712178
[13]	H. Lee, I. Kang, Neural algorithm for solving differential equations, J. Comput. Phys., 91 (1990), 110–131. https://dx.doi.org/10.1016/0021-9991(90)90007-N doi: 10.1016/0021-9991(90)90007-N
[14]	H. Lee, On the contraction properties for weak solutions to linear elliptic equations with $L^2$-drifts of negative divergence, Proc. Amer. Math. Soc., 152 (2024), 2051–2068, https://dx.doi.org/10.1090/proc/16672 doi: 10.1090/proc/16672
[15]	S. Mishra, R. Molinaro, Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs, IMA J. Numer. Anal., 42 (2022), 981–1022. https://dx.doi.org/10.1093/imanum/drab032 doi: 10.1093/imanum/drab032
[16]	S. Mishra, R. Molinaro, Estimates on the generalization error of physics-informed neural networks for approximating PDEs, IMA J. Numer. Anal., 43 (2023), 1–43. https://dx.doi.org/10.1093/imanum/drab093 doi: 10.1093/imanum/drab093
[17]	J. Müller, M. Zeinhofer, Notes on exact boundary values in residual minimisation, Proc. Math. Sci. Mach. Learn., 190 (2022), 231–240.
[18]	M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), 686–707. https://dx.doi.org/10.1016/j.jcp.2018.10.045 doi: 10.1016/j.jcp.2018.10.045
[19]	H. G. Roos, M. Stynes, L. Tobiska, Robust numerical methods for singularly perturbed differential equations: convection-diffusion-reaction and flow problems, Heidelberg: Springer Berlin, 2008. https://dx.doi.org/10.1007/978-3-540-34467-4
[20]	W. Rudin, Principles of mathematical analysis, 3 Eds., New York: McGraw-Hill, 1976.
[21]	E. K. Ryu, Infinitely large neural networks, Lecture Notes in Mathematics, Research Institute of Mathematics, Number 58, 2023.
[22]	Y. Shin, J. Darbon, G. E. Karniadakis, On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs, Commun. Comput. Phys., 28 (2020), 2042–2074. https://dx.doi.org/10.4208/cicp.OA-2020-0193 doi: 10.4208/cicp.OA-2020-0193
[23]	Y. Shin, Z. Zhang, G. E. Karniadakis, Error estimates of residual minimization using neural networks for linear PDEs, J. Mach. Learn. Model. Comput., 4 (2023), 73–101. https://dx.doi.org/10.1615/JMachLearnModelComput.2023050411 doi: 10.1615/JMachLearnModelComput.2023050411
[24]	H. Son, J. Jang, W. Han, H. Hwang, Sobolev training for physics-informed neural networks, Commun. Math. Sci., 21 (2023), 1679–1705. https://dx.doi.org/10.4310/CMS.2023.v21.n6.a11 doi: 10.4310/CMS.2023.v21.n6.a11
[25]	N. Yadav, A. Yadav, M. Kumar, An introduction to neural network methods for differential equations, SpringerBriefs in Applied Sciences and Technology, Dordrecht: Springer, 2015. https://dx.doi.org/10.1007/978-94-017-9816-7
[26]	J. Yoo, J. Kim, M. Gim, H. Lee, Error estimates of Physics-Informed Neural Networks for initial value problems, J. Korean Soc. Ind. Appl. Math., 28 (2024), 33-58. https://dx.doi.org/10.12941/jksiam.2024.28.033 doi: 10.12941/jksiam.2024.28.033
[27]	M. Zeinhofer, R. Masri, K. A. Mardal, A unified framework for the error analysis of physics-informed neural networks, 2024. https://doi.org/10.48550/arXiv.2311.00529

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)