Local polynomial fitting exhibits numerous compelling statistical properties, particularly within the intricate realm of multivariate analysis. However, as functional data analysis gains prominence as a dynamic and pertinent field in data science, the exigency arises for the formulation of a specialized theory tailored to local polynomial fitting. We explored the intricate task of estimating the regression function operator and its partial derivatives for stationary mixing random processes, denoted as $ (Y_i, X_i) $, using local higher-order polynomial fitting. Our key contributions include establishing the joint asymptotic normality of the estimates for both the regression function and its partial derivatives, specifically in the context of strongly mixing processes. Additionally, we provide explicit expressions for the bias and the variance-covariance matrix of the asymptotic distribution. Demonstrating uniform strong consistency over compact subsets, along with delineating the rates of convergence, we substantiated these results for both the regression function and its partial derivatives. Importantly, these findings rooted in reasonably broad conditions that underpinned the underlying models. To demonstrate practical applicability, we leveraged our results to compute pointwise confidence regions. Finally, we extended our ideas to the nonparametric conditional distribution, and obtained its limiting distribution.
Citation: Oussama Bouanani, Salim Bouzebda. Limit theorems for local polynomial estimation of regression for functional dependent data[J]. AIMS Mathematics, 2024, 9(9): 23651-23691. doi: 10.3934/math.20241150
Local polynomial fitting exhibits numerous compelling statistical properties, particularly within the intricate realm of multivariate analysis. However, as functional data analysis gains prominence as a dynamic and pertinent field in data science, the exigency arises for the formulation of a specialized theory tailored to local polynomial fitting. We explored the intricate task of estimating the regression function operator and its partial derivatives for stationary mixing random processes, denoted as $ (Y_i, X_i) $, using local higher-order polynomial fitting. Our key contributions include establishing the joint asymptotic normality of the estimates for both the regression function and its partial derivatives, specifically in the context of strongly mixing processes. Additionally, we provide explicit expressions for the bias and the variance-covariance matrix of the asymptotic distribution. Demonstrating uniform strong consistency over compact subsets, along with delineating the rates of convergence, we substantiated these results for both the regression function and its partial derivatives. Importantly, these findings rooted in reasonably broad conditions that underpinned the underlying models. To demonstrate practical applicability, we leveraged our results to compute pointwise confidence regions. Finally, we extended our ideas to the nonparametric conditional distribution, and obtained its limiting distribution.
[1] | I. M. Almanjahie, S. Bouzebda, Z. C. Elmezouar, A. Laksaci, The functional $k$NN estimator of the conditional expectile: Uniform consistency in number of neighbors, Statist. Risk Model., 38 (2021), 47–63. https://doi.org/10.1515/strm-2019-0029 doi: 10.1515/strm-2019-0029 |
[2] | I. M. Almanjahie, S. Bouzebda, Z. Kaid, A. Laksaci, Nonparametric estimation of expectile regression in functional dependent data, J. Nonparametr. Stat., 34 (2022), 250–281. https://doi.org/10.1080/10485252.2022.2027412 doi: 10.1080/10485252.2022.2027412 |
[3] | I. M. Almanjahie, S. Bouzebda, Z. Kaid, A. Laksaci, The local linear functional $k$NN estimator of the conditional expectile: Uniform consistency in number of neighbors, Metrika, 2024. https://doi.org/10.1007/s00184-023-00942-0 |
[4] | T. W. Anderson, An introduction to multivariate statistical analysis, 3 Eds., John Wiley & Sons, Inc., 1958. |
[5] | G. Aneiros, P. Vieu, Partial linear modeling with multi-functional covariates, Comput. Stat., 30 (2015), 647–671. https://doi.org/10.1007/s00180-015-0568-8 doi: 10.1007/s00180-015-0568-8 |
[6] | S.Attaoui, B. Bentata, S. Bouzebda, A. Laksaci, The strong consistency and asymptotic normality of the kernel estimator type in functional single index model in presence of censored data, AIMS Mathematics, 9 (2024), 7340–7371. https://doi.org/10.3934/math.2024356 doi: 10.3934/math.2024356 |
[7] | S. Ayad, A. Laksaci, S. Rahmani, R. Rouane, On the local linear modelization of the conditional density for functional and ergodic data, Metron, 78 (2020), 237–254. https://doi.org/10.1007/s40300-020-00174-6 doi: 10.1007/s40300-020-00174-6 |
[8] | A. Baíllo, A. Grané, Local linear regression for functional predictor and scalar response, J. Multivariate Anal., 100 (2009), 102–111. https://doi.org/10.1016/j.jmva.2008.03.008 doi: 10.1016/j.jmva.2008.03.008 |
[9] | J. Barrientos-Marin, F. Ferraty, P. Vieu, Locally modelled regression and functional data, J. Nonparametr. Stat., 22 (2010), 617–632. https://doi.org/10.1080/10485250903089930 doi: 10.1080/10485250903089930 |
[10] | R. E. Bellman, Adaptive control processes: A guided tour, Princeton University Press, 1961. |
[11] | K. Benhenni, A. H. Hassan, Y. Su, Local polynomial estimation of regression operators from functional data with correlated errors, J. Multivariate Anal., 170 (2019), 80–94. https://doi.org/10.1016/j.jmva.2018.10.008 doi: 10.1016/j.jmva.2018.10.008 |
[12] | N.-E. Berrahou, S. Bouzebda, L. Douge, Functional uniform-in-bandwidth moderate deviation principle for the local empirical processes involving functional data, Math. Meth. Stat., 33 (2024), 26–69. https://doi.org/10.3103/S1066530724700030 doi: 10.3103/S1066530724700030 |
[13] | D. Bosq, Linear processes in function spaces, In: Lecture notes in statistics, New York: Springer, 2000. https://doi.org/10.1007/978-1-4612-1154-9 |
[14] | O. Bouanani, A. Laksaci, M. Rachdi, S. Rahmani, Asymptotic normality of some conditional nonparametric functional parameters in high-dimensional statistics, Behaviormetrika, 46 (2019), 199–233. https://doi.org/10.1007/s41237-018-0057-9 doi: 10.1007/s41237-018-0057-9 |
[15] | S. Bouzebda, Limit theorems in the nonparametric conditional single-index u-processes for locally stationary functional random fields under stochastic sampling design, Mathematics, 12 (2024), 1996. https://doi.org/10.3390/math12131996 doi: 10.3390/math12131996 |
[16] | S. Bouzebda, Weak convergence of the conditional single index $U $-statistics for locally stationary functional time series, AIMS Mathematics, 9 (2024), 14807–14898. https://doi.org/10.3934/math.2024720 doi: 10.3934/math.2024720 |
[17] | S. Bouzebda, B. Nemouchi, Uniform consistency and uniform in bandwidth consistency for nonparametric regression estimates and conditional $U$-statistics involving functional data, J. Nonparametr. Stat., 32 (2020), 452–509. https://doi.org/10.1080/10485252.2020.1759597 doi: 10.1080/10485252.2020.1759597 |
[18] | S. Bouzebda, B. Nemouchi, Weak-convergence of empirical conditional processes and conditional $U$-processes involving functional mixing data, Stat. Inference Stoch. Process., 26 (2022), 33–88. https://doi.org/10.1007/s11203-022-09276-6 doi: 10.1007/s11203-022-09276-6 |
[19] | S. Bouzebda, A. Nezzal, Uniform consistency and uniform in number of neighbors consistency for nonparametric regression estimates and conditional $U$-statistics involving functional data, Jpn. J. Stat. Data Sci., 5 (2022), 431–533. https://doi.org/10.1007/s42081-022-00161-3 doi: 10.1007/s42081-022-00161-3 |
[20] | S. Bouzebda, A. Nezzal, Uniform in number of neighbors consistency and weak convergence of $k$NN empirical conditional processes and $k$NN conditional $U$-processes involving functional mixing data, AIMS Mathematics, 9 (2024), 4427–4550. https://doi.org/10.3934/math.2024218 doi: 10.3934/math.2024218 |
[21] | S. Bouzebda, A. Laksaci, M. Mohammedi, Single index regression model for functional quasi-associated time series data, REVSTAT Stat. J., 20 (2022), 605–631. http://dx.doi.org/10.57805/revstat.v20i5.391 doi: 10.57805/revstat.v20i5.391 |
[22] | S. Bouzebda, A. Laksaci, M. Mohammedi, The $k$-nearest neighbors method in single index regression model for functional quasi-associated time series data, Rev. Mat. Complut., 36 (2023), 361–391. https://doi.org/10.1007/s13163-022-00436-z doi: 10.1007/s13163-022-00436-z |
[23] | S. Bouzebda, S. Nezzal, T. Zari, Uniform consistency for functional conditional u-statistics using delta-sequences, Mathematics, 11 (2023), 161. https://doi.org/10.3390/math11010161 doi: 10.3390/math11010161 |
[24] | R. C. Bradley, Introduction to strong mixing conditions, Kendrick Press, 2007. |
[25] | J. Camacho, J. Pico, A. Ferrer, Data understanding with PCA: Structural and variance information plots, Chemometr. Intell. Lab. Syst., 100 (2010), 48–56. https://doi.org/10.1016/j.chemolab.2009.10.005 doi: 10.1016/j.chemolab.2009.10.005 |
[26] | H. Cartan, Calcul différentiel, 1967. |
[27] | D. Chen, P. Hall, H. Müller, Single and multiple index functional regression models with nonparametric link, Ann. Statist., 39 (2011), 1720–1747. http://dx.doi.org/10.1214/11-AOS882 doi: 10.1214/11-AOS882 |
[28] | M. Y. Cheng, J. Fan, J. S. Marron, On automatic boundary corrections, Ann. Statist., 25 (1997), 1691–1708. https://doi.org/10.1214/aos/1031594737 doi: 10.1214/aos/1031594737 |
[29] | Z. Chikr-Elmezouar, I. M. Almanjahie, A. Laksaci, M. Rachdi, FDA: strong consistency of the $k$NN local linear estimation of the functional conditional density and mode, J. Nonparametr. Stat., 31 (2019), 175–195. https://doi.org/10.1080/10485252.2018.1538450 doi: 10.1080/10485252.2018.1538450 |
[30] | J. Dauxois, A. Pousse, Y. Romain, Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference, J. Multivariate Anal., 12 (1982), 136–154. https://doi.org/10.1016/0047-259X(82)90088-4 doi: 10.1016/0047-259X(82)90088-4 |
[31] | J. Demongeot, A. Laksaci, M. Rachdi, S. Rahmani, On the local linear modelization of the conditional distribution for functional data, Sankhya A, 76 (2014), 328–355. https://doi.org/10.1007/s13171-013-0050-z doi: 10.1007/s13171-013-0050-z |
[32] | J. Demongeot, A. Naceri, A. Laksaci, M. Rachdi, Local linear regression modelization when all variables are curves, Statist. Probab. Lett., 121 (2017), 37–44. https://doi.org/10.1016/j.spl.2016.09.021 doi: 10.1016/j.spl.2016.09.021 |
[33] | J. C. Deville, Méthodes statistiques et numériques de l'analyse harmonique, Annales de l'INSEE, 15 (1974), 3–101. https://doi.org/10.2307/20075177 doi: 10.2307/20075177 |
[34] | S. Didi, S. Bouzebda, Wavelet density and regression estimators for continuous time functional stationary and ergodic processes, Mathematics, 10 (2022), 4356. https://doi.org/10.3390/math10224356 doi: 10.3390/math10224356 |
[35] | Z. C.Elmezouar, F. Alshahrani, I. M. Almanjahie, S. Bouzebda, Z. Kaid, A. Laksaci, Strong consistency rate in functional single index expectile model for spatial data, AIMS Mathematics, 9 (2024), 5550–5581. https://doi.org/10.3934/math.2024269 doi: 10.3934/math.2024269 |
[36] | R. L. Eubank, Nonparametric regression and spline smoothing, 2 Eds., CRC Press, 2014. https://doi.org/10.1201/9781482273144 |
[37] | J. Fan, I. Gijbels, Local polynomial modeling and its applications, Chapman & Hall, 1996. |
[38] | F. Ferraty, P. Vieu, Nonparametric functional data analysis, New York: Springer, 2006. https://doi.org/10.1007/0-387-36620-2 |
[39] | F. Ferraty, A. Laksaci, P. Vieu, Estimating some characteristics of the conditional distribution in nonparametric functional models, Stat. Infer. Stoch. Process., 9 (2006), 47–76. https://doi.org/10.1007/s11203-004-3561-3 doi: 10.1007/s11203-004-3561-3 |
[40] | F. Ferraty, A. Mas, P. Vieu, Nonparametric regression on functional data: inference and practical aspects, Aust. N. Z. J. Stat., 49 (2007), 267–286. https://doi.org/10.1111/j.1467-842X.2007.00480.x doi: 10.1111/j.1467-842X.2007.00480.x |
[41] | F. Ferraty, A. Laksaci, A. Tadj, P. Vieu, Rate of uniform consistency for nonparametric estimates with functional variables, J. Statist. Plann. Inference, 140 (2010), 335–352. https://doi.org/10.1016/j.jspi.2009.07.019 doi: 10.1016/j.jspi.2009.07.019 |
[42] | F. Ferraty, A. Laksaci, A. Tadj, P. Vieu, Estimation de la fonction de régression pour variable explicative et réponse fonctionnelles dépendantes, C. R. Acad. Sci. Paris, Ser. I, 350 (2012), 717–720. https://doi.org/10.1016/j.crma.2012.07.014 doi: 10.1016/j.crma.2012.07.014 |
[43] | L. Ferré, A. F. Yao, Smoothed functional inverse regression, Statist. Sinica, 15 (2005), 665–683. |
[44] | A. Goia, P. Vieu, A partitioned single functional index model, Comput. Stat., 30 (2015), 673–692. https://doi.org/10.1007/s00180-014-0530-1 doi: 10.1007/s00180-014-0530-1 |
[45] | A. Goia, P. Vieu, An introduction to recent advances in high/infinite dimensional statistics, J. Multivariate Anal., 146 (2016), 1–6. https://doi.org/10.1016/j.jmva.2015.12.001 doi: 10.1016/j.jmva.2015.12.001 |
[46] | P. J. Green, B. W. Silverman, Nonparametric regression and generalized linear models, New York: Chapman and Hall/CRC, 1993. https://doi.org/10.1201/b15710 |
[47] | C. Gu, Smoothing spline ANOVA models, New York: Springer, 2015. https://doi.org/10.1007/978-1-4614-5369-7 |
[48] | L. Horváth, P. Kokoszka, Inference for functional data with applications, New York: Springer, 2014. https://doi.org/10.1007/978-1-4614-3655-3 |
[49] | L. Horváth, G. Rice, An introduction to functional data analysis and a principal component approach for testing the equality of mean curves, Rev. Mat. Complut., 28 (2015), 505–548. https://doi.org/10.1007/s13163-015-0169-7 doi: 10.1007/s13163-015-0169-7 |
[50] | G. M. James, T. J. Hastie, C. A. Sugar, Principal component models for sparse functional data, Biometrika, 87 (2000), 587–602. https://doi.org/10.1093/biomet/87.3.587 doi: 10.1093/biomet/87.3.587 |
[51] | A. Khaleghi, G. Lugosi, Inferring the mixing properties of an ergodic process, arXiv: 2106.07054, 2021. https://doi.org/10.48550/arXiv.2106.07054 |
[52] | A. N. Kolmogorov, V. M. Tihomirov, $\varepsilon $-entropy and $\varepsilon $-capacity of sets in function spaces, Uspehi Mat. Nauk, 14 (1959), 3–86. |
[53] | J. Kuelbs, W. V. Li, Metric entropy and the small ball problem for Gaussian measures, J. Funct. Anal., 116 (1993), 133–157. https://doi.org/10.1006/jfan.1993.1107 doi: 10.1006/jfan.1993.1107 |
[54] | W. V. Li, Q. M. Shao, Gaussian processes: inequalities, small ball probabilities and applications, Handbook of Statist., 19 (2001), 533–597. https://doi.org/10.1016/S0169-7161(01)19019-X doi: 10.1016/S0169-7161(01)19019-X |
[55] | H. Y. Liang, J. I. Baek, Asymptotic normality of conditional density estimation with left-truncated and dependent data, Stat. Papers, 57 (2016), 1–20. https://doi.org/10.1007/s00362-014-0635-1 doi: 10.1007/s00362-014-0635-1 |
[56] | E. Liebscher, Estimation of the density and the regression function under mixing conditions, Statist. Decisions, 19 (2001), 9–26. https://doi.org/10.1524/strm.2001.19.1.9 doi: 10.1524/strm.2001.19.1.9 |
[57] | N. Ling, P. Vieu, Nonparametric modeling for functional data: Selected survey and tracks for future, Statistics, 52 (2018), 934–949. https://doi.org/10.1080/02331888.2018.1487120 doi: 10.1080/02331888.2018.1487120 |
[58] | S. Lv, H. Lin, H. Lian, J. Huang, Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space, Ann. Statist., 46 (2018), 781–813. https://doi.org/10.1214/17-AOS1567 doi: 10.1214/17-AOS1567 |
[59] | E. Masry, Multivariate local polynomial regression for time series: uniform strong consistency and rates, J. Time Ser. Anal., 17 (1996), 571–599. https://doi.org/10.1111/j.1467-9892.1996.tb00294.x doi: 10.1111/j.1467-9892.1996.tb00294.x |
[60] | E. Masry, Multivariate regression estimation local polynomial fitting for time series, Stochastic Process. Appl., 65 (1996), 81–101. https://doi.org/10.1016/S0304-4149(96)00095-6 doi: 10.1016/S0304-4149(96)00095-6 |
[61] | E. Masry, Multivariate regression estimation of continuous-time processes from sampled data: Local polynomial fitting approach, IEEE Trans. Inform. Theory, 45 (1999), 1939–1953. https://doi.org/10.1109/18.782116 doi: 10.1109/18.782116 |
[62] | E. Masry, Nonparametric regression estimation for dependent functional data: asymptotic normality, Stochastic Process. Appl., 115 (2005), 155–177. https://doi.org/10.1016/j.spa.2004.07.006 doi: 10.1016/j.spa.2004.07.006 |
[63] | E. Masry, Local polynomial estimation of regression functions for mixing processes, Scand. J. Statist., 24 (1997), 165–179. https://doi.org/10.1111/1467-9469.00056 doi: 10.1111/1467-9469.00056 |
[64] | E. Masry, J. Mielniczuk, Local linear regression estimation for time series with long-range dependence, Stochastic Process. Appl., 82 (1999), 173–193. https://doi.org/10.1016/S0304-4149(99)00015-0 doi: 10.1016/S0304-4149(99)00015-0 |
[65] | D. J. McDonald, C. R. Shalizi, M. Schervish, Estimating beta-mixing coefficients via histograms, Electron. J. Stat., 9 (2015), 2855–2883. http://dx.doi.org/10.1214/15-EJS1094 doi: 10.1214/15-EJS1094 |
[66] | M. Mohammedi, S. Bouzebda, A. Laksaci, The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data, J. Multivariate Anal., 181 (2021), 104673. https://doi.org/10.1016/j.jmva.2020.104673 doi: 10.1016/j.jmva.2020.104673 |
[67] | H. Müller, Peter hall, functional data analysis and random objects, Ann. Statist., 44 (2016), 1867–1887. https://doi.org/10.1214/16-AOS1492 doi: 10.1214/16-AOS1492 |
[68] | E. A.Nadaraja, On estimating regression, Theory Probab. Appl., 9 (1964), 141–142. https://doi.org/10.1137/1109020 |
[69] | T. Nicoleris, Y. G. Yatracos, Rates of convergence of estimates, Kolmogorov's entropy and the dimensionality reduction principle in regression, Ann. Statist., 25 (1997), 2493–2511. https://doi.org/10.1214/aos/1030741082 doi: 10.1214/aos/1030741082 |
[70] | E. Parzen, On estimation of a probability density function and mode, Ann. Math. Statist., 33 (1962), 1065–1076. |
[71] | J. Q. Shi, T. Choi, Gaussian process regression analysis for functional data, New York: Chapman and Hall/CRC, 2011. https://doi.org/10.1201/b11038 |
[72] | M. Rachdi, A. Laksaci, J. Demongeot, A. Abdali, F. Madani, Theoretical and practical aspects of the quadratic error in the local linear estimation of the conditional density for functional data, Comput. Statist. Data Anal., 73 (2014), 53–68. https://doi.org/10.1016/j.csda.2013.11.011 doi: 10.1016/j.csda.2013.11.011 |
[73] | M. Rachdi, A. Laksaci, Z. Kaid, A. Benchiha, F. A. Al-Awadhi, $k$-nearest neighbors local linear regression for functional and missing data at random, Statist. Neerlandica, 75 (2021), 42–65. https://doi.org/10.1111/stan.12224 doi: 10.1111/stan.12224 |
[74] | J. O. Ramsay, B. W. Silverman, Functional data analysis, In: Springer series in statistics, New York: Springer, 2005. https://doi.org/10.1007/b98888 |
[75] | E. Rio, Théorie asymptotique des processus aléatoires faiblement dépendants, In: Mathématiques et applications, Heidelberg: Springer-Verlag Berlin, 31 (2000). |
[76] | M. Rosenblatt, A central limit theorem and a strong mixing condition, Proc. Natl. Acad. Sci., 42 (1956), 43–47. https://doi.org/10.1073/pnas.42.1.43 doi: 10.1073/pnas.42.1.43 |
[77] | D. W. Scott, M. P. Wand, Feasibility of multivariate density estimates, Biometrika, 78 (1991), 197–205. |
[78] | B. W. Silverman, Density estimation for statistics and data analysis, New York: Routledge, 1998. https://doi.org/10.1201/9781315140919 |
[79] | I. Soukarieh, S. Bouzebda, Weak convergence of the conditional $U$-statistics for locally stationary functional time series, Stat. Inference Stoch. Process., 27 (2024), 227–304. https://doi.org/10.1007/s11203-023-09305-y doi: 10.1007/s11203-023-09305-y |
[80] | G. Strang, Wavelet transforms versus Fourier transforms, Bull. Amer. Math. Soc., 28 (1993), 288–305. |
[81] | M. Talagrand, Sharper bounds for Gaussian and empirical processes, Ann. Probab., 22 (1994), 28–76. |
[82] | V. A. Volkonskii, Y. A. Rozanov, Some limit theorems for random functions. Ⅰ, Theory Probab. Appl., 4 (1959), 178–197. https://doi.org/10.1137/1104015 doi: 10.1137/1104015 |
[83] | V. A. Volkonskii, Y. A. Rozanov, Some limit theorems for random functions. Ⅱ, Theory Probab. Appl., 6 (1961), 186–198. https://doi.org/10.1137/1106023 doi: 10.1137/1106023 |
[84] | G. Wahba, Spline models for observational data, In: CBMS-NSF Regional conference series in applied mathematics, Society for industrial and applied mathematics (SIAM), 1990. https://doi.org/10.1137/1.9781611970128 |
[85] | G. S. Watson, Smooth regression analysis, Sankhya, 26 (1964), 359–372. |
[86] | L. Xiao, Asymptotic theory of penalized splines, Electron. J. Statist., 13 (2019), 747–794. https://doi.org/10.1214/19-EJS1541 doi: 10.1214/19-EJS1541 |
[87] | L. Xiao, Asymptotic properties of penalized splines for functional data, Bernoulli, 26 (2020), 2847–2875. https://doi.org/10.3150/20-BEJ1209 doi: 10.3150/20-BEJ1209 |
[88] | F. Yao, H. G. Müller, J. L. Wang, Functional data analysis for sparse longitudinal data, J. Amer. Statist. Assoc., 100 (2005), 577–590. https://doi.org/10.1198/016214504000001745 doi: 10.1198/016214504000001745 |
[89] | M. Yuan, T. T. Cai, A reproducing kernel Hilbert space approach to functional linear regression, Ann. Statist., 38 (2010), 3412–3444. https://doi.org/10.1214/09-AOS772 doi: 10.1214/09-AOS772 |
[90] | E. Zeidler, Nonlinear functional analysis and its applications. I, New York: Springer, 1986. |
[91] | J. Zhang, Analysis of variance for functional data, New York: Chapman and Hall/CRC, 2013, https://doi.org/10.1201/b15005 |
[92] | Z. Zhou, Z. Lin, Asymptotic normality of locally modelled regression estimator for functional data, J. Nonparametr. Stat., 28 (2016), 116–131. https://doi.org/10.1080/10485252.2015.1114112 doi: 10.1080/10485252.2015.1114112 |