A Bernstein polynomial approach of the robust regression

Sihem Semmar; Omar Fetitah; Mohammed Kadi Attouch; Salah Khardani; Ibrahim M. Almanjahie; Sihem Semmar; Omar Fetitah; Mohammed Kadi Attouch; Salah Khardani; Ibrahim M. Almanjahie

doi:10.3934/math.20241554

AIMS Mathematics

2024, Volume 9, Issue 11: 32409-32441. doi: 10.3934/math.20241554

Previous Article Next Article

Research article Special Issues

A Bernstein polynomial approach of the robust regression

1.
Ecole Normale Supérieure d'Oran AMMOUR Ahmed, Oran, Algeria
2.
Laboratory of Statistics and Stochastic Processes, University of Djillali Liabes, BP 89, Sidi Bel Abbes 22000, Algeria
3.
Ecole Superieure en Informatique, Sidi Bel Abbes, Algeria
4.
Faculté des Sciences Tunis, Université El-Manar, Laboratoire de Modélisation Mathématique, Statistique et Analyse Stochastique, Tunis, Tunisia
5.
Department of Mathematics, College of Science, King Khalid University, Abha 62223, Saudi Arabia

Received: 12 August 2024 Revised: 31 October 2024 Accepted: 04 November 2024 Published: 15 November 2024
MSC : 62G05, 62G35

This paper proposes a new family of robust non-parametric estimators for regression functions by applying polynomials to construct a robust regression estimator. Theoretical results and tests on simulated and real data sets validate the efficiency and practicality of the approach. Moreover, some of its asymptotic properties are discussed and demonstrated. Experimental studies are conducted to compare this new approach with the Bernstein-Nadaraya-Watson estimator and the Nadaraya-Watson estimators. Some simulations are performed to illustrate that our robust estimator has the lowest average integrated squared error ($ \overline{AISE} $). In the end, real data is utilized to assess the performance of conventional and newly presented robust regression algorithms regarding their ability to handle sensitivity to outliers.
- nonparametric regression,
- Bernstein polynomial,
- robust estimation,
- asymptotic normality,
- Nadaraya-Watson estimators
Citation: Sihem Semmar, Omar Fetitah, Mohammed Kadi Attouch, Salah Khardani, Ibrahim M. Almanjahie. A Bernstein polynomial approach of the robust regression[J]. AIMS Mathematics, 2024, 9(11): 32409-32441. doi: 10.3934/math.20241554

Related Papers:

Abstract

This paper proposes a new family of robust non-parametric estimators for regression functions by applying polynomials to construct a robust regression estimator. Theoretical results and tests on simulated and real data sets validate the efficiency and practicality of the approach. Moreover, some of its asymptotic properties are discussed and demonstrated. Experimental studies are conducted to compare this new approach with the Bernstein-Nadaraya-Watson estimator and the Nadaraya-Watson estimators. Some simulations are performed to illustrate that our robust estimator has the lowest average integrated squared error ($ \overline{AISE} $). In the end, real data is utilized to assess the performance of conventional and newly presented robust regression algorithms regarding their ability to handle sensitivity to outliers.

References

[1]	R. J. Huber, Robust estimation of a location parmeter, Ann. Math. Statist., 35 (1964), 73–101. https://doi.org/10.1214/aoms/1177703732 doi: 10.1214/aoms/1177703732
[2]	P. Robinson, Robust nonparametric autoregression, In: Robust and nonlinear time series analysis, New York: Springer, 26 (1986), 247–255. https://doi.org/10.1007/978-1-4615-7821-5_14
[3]	G. Collomb, W. Härdle, Strong uniform convergence rates in robust nonparametric time series analysis and prediction: Kernel regression estimation from dependent observations, Stoch. Process. Appl., 23 (1986), 77–89. https://doi.org/10.1016/0304-4149(86)90017-7 doi: 10.1016/0304-4149(86)90017-7
[4]	G. Boente, R. Fraiman, Robust nonparametric regression estimation for dependent observations, Ann. Statist., 17 (1989), 1242–1256. https://doi.org/10.1214/aos/1176347266 doi: 10.1214/aos/1176347266
[5]	G. Boente, R. Fraiman, Asymptotic distribution of robust estimators for nonparametric models from mixing processes, Ann. Statist., 18 (1990), 891–906. https://doi.org/10.1214/aos/1176347631 doi: 10.1214/aos/1176347631
[6]	J. Fan, T. C. Hu, Y. K. Truong, Robust non-parametric function estimation, Scand. J. Statist., 21 (1994), 433–446.
[7]	N. Laïb, E. Ould-Saïd, A robust nonparametric estimation of the autoregression function under ergodic hypothesis, Canad. J. Statist., 28 (2000), 817–828. https://doi.org/10.2307/3315918 doi: 10.2307/3315918
[8]	G. Boente, D. Rodriguez, Robust estimators of high order derivatives of regression function, Statist. Probab. Lett., 76 (2006), 1335–1344. https://doi.org/10.1016/j.spl.2006.01.011 doi: 10.1016/j.spl.2006.01.011
[9]	R. A. Vitale, A Bernstein polynomial approach to density function estimation, In: Statistical inference and related topics, 1975, 87–99. https://doi.org/10.1016/B978-0-12-568002-8.50011-2
[10]	S. Petrone, Bayesian density estimation using bernstein polynomials, Canad. J. Statist., 27 (1999), 105–126. https://doi.org/10.2307/3315494 doi: 10.2307/3315494
[11]	S. Petrone, Random Bernstein polynomials, Scand. J. Statist., 26 (1999), 373–393. https://doi.org/10.1111/1467-9469.00155 doi: 10.1111/1467-9469.00155
[12]	G. J. Babu, A. J. Canty, Y. P. Chaubey, Application of Bernstein polynomials for smooth estimation of a distribution and density function, J. Statist. Plann. Inference, 105 (2002), 377–392. https://doi.org/10.1016/S0378-3758(01)00265-8 doi: 10.1016/S0378-3758(01)00265-8
[13]	S. Petrone, L. Wasserman, Consistency of Bernstein polynomial posteriors, J. R. Stat. Soc. Ser. B Stat. Methodol., 64 (2002), 79–100. https://doi.org/10.1111/1467-9868.00326 doi: 10.1111/1467-9868.00326
[14]	Y. Kakizawa, Bernstein polynomial probability density estimation, J. Nonparametr. Stat., 16 (2004), 709–729. https://doi.org/10.1080/1048525042000191486 doi: 10.1080/1048525042000191486
[15]	F. Ouimet, Asymptotic properties of Bernstein estimators on the simplex, J. Multivariate Anal., 185 (2021), 104784. https://doi.org/10.1016/j.jmva.2021.104784 doi: 10.1016/j.jmva.2021.104784
[16]	M. Belalia, T. Bouezmarni, A. Leblanc, Smooth conditional distribution estimators using Bernstein polynomials, Comput. Statist. Data Anal., 111 (2017), 166–182. https://doi.org/10.1016/j.csda.2017.02.005 doi: 10.1016/j.csda.2017.02.005
[17]	S. Khardani. A Bernstein polynomial approach to the estimation of a distribution function and quantiles under censorship model, Comm. Statist. Theory Methods, 53 (2024), 5673–5686. https://doi.org/10.1080/03610926.2023.2228948 doi: 10.1080/03610926.2023.2228948
[18]	M. B. Priestley, M. T. Chao, Non-parametric function fitting, J. R. Stat. Soc. Ser. B Stat. Methodol, 34 (1972), 385–392. https://doi.org/10.1111/j.2517-6161.1972.tb00916.x doi: 10.1111/j.2517-6161.1972.tb00916.x
[19]	A. Tenbusch, Nonparametric curve estimation with Bernstein estimates, Metrika, 45 (1997), 1–30. https://doi.org/10.1007/BF02717090 doi: 10.1007/BF02717090
[20]	B. M. Brown, S. X. Chen, Beta-Bernstein smoothing for regression curves with compact support, Scand. J. Statist., 26 (1999), 47–59. https://doi.org/10.1111/1467-9469.00136 doi: 10.1111/1467-9469.00136
[21]	N. Choudhuri, S. Ghosal, A. Roy, Bayesian estimation of the spectral density of a time series, J. Amer. Statist. Assoc., 99 (2004), 1050–1059. https://doi.org/10.1198/016214504000000557 doi: 10.1198/016214504000000557
[22]	I. S. Chang, C. A. Hsiung, Y. J. Wu, C. C. Yang, Bayesian survival analysis using Bernstein polynomials, Scand. J. Statist., 32 (2005), 447–466. https://doi.org/10.1111/j.1467-9469.2005.00451.x doi: 10.1111/j.1467-9469.2005.00451.x
[23]	Y. Kakizawa, A note on generalized Bernstein polynomial density estimators, Stat. Methodol., 8 (2011), 136–153. https://doi.org/10.1016/j.stamet.2010.08.004 doi: 10.1016/j.stamet.2010.08.004
[24]	Y. Slaoui, A. Jmaei, Recursive density estimators based on Robbins-Monro's scheme and using Bernstein polynomials, Stat. Interface, 12 (2019), 439–455. https://doi.org/10.4310/19-SII561 doi: 10.4310/19-SII561
[25]	C. J. Stone, Nonparametric M-regression with free knot splines, J. Statist. Plann. Inference, 130 (2005), 183–206. https://doi.org/10.1016/j.jspi.2003.05.002 doi: 10.1016/j.jspi.2003.05.002
[26]	A. Leblanc, A bias-reduced approach to density estimation using Bernstein polynomials, J. Nonparametr. Stat., 22 (2010), 459–475. https://doi.org/10.1080/10485250903318107 doi: 10.1080/10485250903318107
[27]	A. Leblanc, On estimating distribution function using Bernstein polynomials, Ann. Inst. Stat. Math., 64 (2012), 919–943. https://doi.org/10.1007/s10463-011-0339-4 doi: 10.1007/s10463-011-0339-4
[28]	Y. Slaoui, A. Jmaei, Recursive and non-recursive regression estimators using Bernstein polynomials, Theory Stoch. Process., 26 (2022), 60–95. https://doi.org/10.37863/tsp-2899660400-77 doi: 10.37863/tsp-2899660400-77
[29]	A. Tenbusch, Two-dimensional Bernstein polynomial density estimation, Metrika, 41 (1994), 233–253. https://doi.org/10.1007/BF01895321 doi: 10.1007/BF01895321
[30]	E. A. Nadaraya, On estimating regression, Theory Probab. Appl., 9 (1964), 141–142. https://doi.org/10.1137/1109020 doi: 10.1137/1109020
[31]	G. S. Watson, Smooth regression analysis, Sankhya, 26 (1975), 359–372.
[32]	G. Boente, R. Fraiman, Robust nonparametric regression estimation for dependent observations, Ann. Statist., 17 (1989), 1242–1256. https://doi.org/10.1214/aos/1176347266 doi: 10.1214/aos/1176347266
[33]	L. Aït Hennani, M. Lemdani, E. O. Saïd, Robust regression analysis for a censored response and functional regressors, J. Nonparametr. Stat., 31 (2019), 221–243. https://doi.org/10.1080/10485252.2018.1546386 doi: 10.1080/10485252.2018.1546386
[34]	M. Attouch, A. Laksaci, E. Ould-Saïd, Asymptotic distribution of robust estimator for functional nonparametric models, Comm. Statist. Theory Methods, 38 (2009), 1317–1335. https://doi.org/10.1080/03610920802422597 doi: 10.1080/03610920802422597
[35]	M. K. Attouch, A. Laksaci, E. Ould-Saïd, Robust regression for functional time series data, J. Japan Statist. Soc., 42 (2012), 125–143. https://doi.org/10.14490/jjss.42.125 doi: 10.14490/jjss.42.125
[36]	W. Feller, An introduction to probability theory and its applications, 2nd Eds., Chapman & Hall, 1958.
[37]	M. Loève, Probability theory, New York: Springer, 1977. https://doi.org/10.1007/978-1-4684-9464-8

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)