Optimal clustering by merge-based branch-and-bound

1.   Introduction

In recent decades, there has been a growing interest in the statistical study of functional random variables, which are values in infinite-dimensional spaces. Simply put, functional data analysis (FDA) typically deals with statistical problems where the data consists of a sample of $n$ functions $x_1 = x_1(t), \ldots, x_n = x_n(t)$, defined on a compact interval of the real line $[0, 1]$. FDA focuses on statistical issues, often referred to as "inference in stochastic processes", where sample information is derived from a partial trajectory $(x(t), t \in [0, T])$ of a stochastic process $(\{X(t), t \geq 0\})$. In this context, the duration $(T)$ of the observation interval acts as the sample size $(n)$. This increasing interest has brought to light several statistical challenges associated with functional random variables. The drive to explore this field has been fueled by the increasing availability of high-resolution temporal and spatial data. This trend is particularly notable in fields such as meteorology, medicine, satellite imaging, and various other scientific disciplines. Consequently, the statistical modeling of this data, viewed as stochastic functions, has led to numerous complex theoretical and computational research questions. To gain a comprehensive understanding of both the theoretical and practical components of functional data analysis, it is recommended that the reader consult the monographs authored by ^[13] for linear models about random variables that assume values in a Hilbert space, and ^[74] for scalar-on-function and function-on-function linear models, as well as functional principal component analysis and parametric discriminant analysis. Ferraty and Vieu ^[38] primarily concentrated on nonparametric techniques, particularly kernel-based estimation, for scalar-on-function nonlinear regression models. These methods were further expanded to encompass classification and discrimination analysis. In their study, ^[48] examined the extension of various statistical concepts, including goodness-of-fit tests, portmanteau tests, and change detection, to the functional data framework. Zhang ^[91] conducted a study on the analysis of variance for functional data, while ^[71] primarily investigated regression analysis for Gaussian processes. Various semi-parametric models have been explored in the literature, such as functional single index models ^[44], projection pursuit models ^[27], partial linear models ^[5], and functional sliced inverse regression ^[43]. Additional studies on functional data modeling and analysis have been documented in the following sources: ^{[45,49,57,67]} and for recent references ^{[1,2,15,17,18,19,20,21,22,66,79]}. One of the most used estimators is the Nadaraya-Watson ^[68,85] estimator. From a function approximation perspective, the Nadaraya-Watson estimator employs local constant approximations. As highlighted by the numerical analyst ^[80], "Through all of scientific computing runs this common theme: Increase the accuracy at least to second order. What this means is: Get the linear term right". In other words, local constant approximations are often inadequate, and local linear fits are preferable. Local polynomial fitting emerges as an attractive method from both theoretical and practical standpoints. Local polynomial fitting offers several advantages, as noted in ^[37]. It adapts to various design types, including random and fixed designs, highly clustered, and nearly uniform designs. Moreover, boundary effects are absent: the bias at the boundary remains of the same order as in the interior, without requiring specific boundary kernels. This is a significant departure from other existing methods. With local polynomial fitting, no boundary modifications are necessary, which is particularly beneficial in multidimensional situations where the boundary can involve a substantial number of data points (see ^[28,78]). Boundary modifications in higher dimensions pose a considerable challenge. The pivotal method proposed by ^[60] involved utilizing local higher-order polynomial fitting to estimate the function $m$ and its partial derivatives. The author demonstrated mean-square convergence and joint asymptotic normality (see also ^[61]). This research holds significant implications for estimating widely used ARCH time series. Masry ^[59] established the strong consistency of the regression estimator and its partial derivatives up to a fixed total order $p$, obtaining sharp rates of almost sure convergence that are uniform over compact sets. In ^[63], the local polynomial estimation of regression functions and their derivatives is examined, establishing the joint asymptotic normality of the relevant estimates for both strongly mixing and $\rho$-mixing processes. Masry and Mielniczuk ^[64] considered the nonparametric estimation of a multivariate regression function and its derivatives for a regression model with long-range dependent errors. The authors adopted a local linear fitting approach and established the joint asymptotic distributions for the estimators of the regression function and its derivatives.

The entire body of literature on functional data is primarily centered around the theme of local linear estimation. Rachdi et al. ^[73] innovatively amalgamated the $k$-Nearest Neighbors method with the local linear estimation approach to formulate a novel estimator for the regression operator. This innovative approach was tailored for situations where the regressor adopts a functional form, and the response variable, though scalar, is subject to random missing observations. On a parallel note, ^[7] undertook the task of estimating the conditional density function through the application of the local linear methodology. Moreover, Chikr-Elmezouar et al. ^[29] delved into the estimation of the conditional density and mode when confronted with functional covariates. In the realm of local linear regression, ^[8] presented a novel local linear regression estimator and conducted a comprehensive study of its asymptotic behavior. The work by ^[9] and ^[11] introduced a general framework elucidating the local behavior of the regression operator, refer to ^[3] for recent references. The foundational work of ^[92] established mean-squared convergence and asymptotic normality for the local linear estimator. Expanding the scope to instances where both the response and explanatory variables are functionally characterized, ^[32] devised a nonparametric local linear estimator for the regression function. The principal objective of the current investigation is to establish a comprehensive framework for polynomial fitting. The central aim of this paper is to furnish the inaugural comprehensive theoretical validation for the kernel polynomial estimator. This entails determining the uniform consistency rate and discerning the asymptotic distribution. Nonetheless, it becomes evident that addressing this issue transcends a mere amalgamation of ideas from disparate domains. Instead, intricate mathematical derivations are imperative to contend with the challenges posed by estimators in our specific context. This necessitates the adept application of techniques rooted in large sample theory.

The article is structured as follows. Section 2 provides the necessary notation and definitions. The main results are outlined in Section 3. The asymptotic distribution is discussed in Section 3.1, and its application to the confidence regions is presented in Section 3.3. The establishment of uniform almost complete convergence is detailed in Section 3.4. Local linear estimators for the conditional distribution and their asymptotic normality are presented in Section 4. In Section 5, we summarize the findings and highlight remaining open issues. All proofs are deferred to Section 6, with a focus on the most crucial arguments due to the lengthiness of the proofs. Additionally, relevant technical results are provided in the appendix.

2.   Local polynomial regression

We begin by introducing the necessary notation and definitions for the forthcoming sections. Specifically, we recall the definition of the strong mixing property. Let $\mathscr{F}_{i}^{k}(Z)$ denote the $\sigma$-algebra generated by $\{Z_{j} : i \leq j \leq k\}$. The following definition of the strong mixing coefficient is attributed to ^[76]. For further details, refer to ^{[24,75,82,83]}.

Definition 2.1. Let $Z = \{Z_{i}, i = 1, 2, \ldots \}$ be a strictly stationary sequence of random variables. Given a positive integer $n$, define

The sequence $Z$ is said to be $\alpha$-mixing (strong mixing) if the mixing coefficient $\alpha(n) \to 0$ as $n \to \infty$.

The $\alpha$-mixing condition is the weakest among the commonly used mixing conditions and has numerous practical applications, particularly in economics and finance. For example, several time series models, such as ARCH, ARMA, GARMA, and stochastic volatility models, satisfy the $\alpha$-mixing condition.

Remark 2.2. Asymptotic independence, or "mixing", of the data-generating process is a common assumption in statistical learning for time series. For many typical processes, the general forms of various mixing rates are often assumed to be known, but specific coefficients are not. These mixing assumptions are rarely tested, and there are no established methods for estimating mixing coefficients from data. In ^[65], an estimator for beta-mixing coefficients based on a single stationary sample path is introduced. Khaleghi and Lugosi ^[51] propose strongly consistent estimators for the $\ell_1$ norm of the sequence of $\alpha$-mixing and $\beta$-mixing coefficients of a stationary ergodic process. These estimators are subsequently used to develop strongly consistent goodness-of-fit hypothesis tests. Specifically, Khaleghi and Lugosi ^[51] develop hypothesis tests to determine whether, under the same summability assumption, the $\alpha$-mixing or $\beta$-mixing coefficients of a process are upper bounded by a given rate function.

We consider a sequence $\{\left(X_i, Y_i\right): i\geq 1\}$ of stationary^* $\alpha-$mixing random copies of the random vector [rv] $(X, Y)$, where $X$ takes its values in some abstract space $\mathfrak{F}$ and $Y$ in $\mathbb R.$ Suppose that $\mathfrak{F}$ is endowed with a semi-metric $d(\cdot, \cdot)$^† defining a topology to measure the proximity between two elements of $\mathfrak{F}$ and which is disconnected from the definition of $X$ to avoid measurability problems. We will consider especially the conditional expectation of $\psi(Y)$ given $X = x$,

^*In the case of Hilbert space-valued elements, strict stationarity is not necessarily required; second-order stationarity suffices. A Hilbert space-valued sequence $\{X_{t}\}_{t \in \mathbb{Z}}$ is second-order (or weakly) stationary if $\mathbb{E}\left\|X_{t}\right\|^{2} < \infty$, $\mathbb{E} X_{t} = \mu$, and $\mathbb{E}\left[(X_{s} - \mu) \otimes (X_{t} - \mu)\right] = \mathbb{E}\left[(X_{s-t} - \mu) \otimes (X_{0} - \mu)\right]$ for all $s, t \in \mathbb{Z}$.

We say that $\{X_{t}\}_{t \in \mathbb{Z}}$ is strictly stationary if the joint distribution of $\{X_{t_{1}}, \ldots, X_{t_{n}}\}$ is the same as the joint distribution of $\{X_{t_{1}+h}, \ldots, X_{t_{n}+h}\}$ for all $t_{1}, \ldots, t_{n} \in \mathbb{Z}$, $n \geq 1$, and $h \geq 1$.

^†A semi-metric (sometimes called pseudo-metric) $d(\cdot, \cdot)$ is a metric which allows $d(x_{1}, x_{2}) = 0$ for some $x_{1}\neq x_{2}$.

whenever this regression function is meaningful. Here and elsewhere, $\psi(\cdot)$ denotes a specified measurable function, which is assumed to be bounded on each compact subinterval of $\mathbb{R}$. Note that we can write

In this work, we study the problem of nonparametric estimation of the regressor function $r(\cdot)$ using, for the first time in the functional data, the local polynomial fitting such that the regressor belongs to an infinite dimensional set. From now on, for the ease of the notation, we set $\psi(y) = y.$ Recall that in functional statistics, there are several approaches to developing the concept of local linear methods. For example (see ^[9]) the linear approximation of $r(\cdot)$ for any $z$ in the neighborhood of $x$ is given by

where the quantities $a = r(x)$ and $b$ are estimated by minimizing the following criterion

where the locating functions $\beta(\cdot, \cdot)$ and $\delta(\cdot, \cdot)$ are defined on $(\mathfrak{F} \times \mathfrak{F})$ and map into $\mathbb{R}$, with $|\delta(\cdot, \cdot)| = d(\cdot, \cdot)$, where $d(\cdot, \cdot)$ is a distance metric. The function $\beta$ refers to the local behavior of our model, while $K(\cdot)$ is a kernel function, and $h_{K} = h_{K, n}$ is the bandwidth or smoothing parameter of the kernel $K(\cdot)$, controlling the size of the local neighborhood and the degree of smoothing. The performance of the estimate depends crucially on the two locating functions, $\beta(\cdot, \cdot)$ and $\delta(\cdot, \cdot)$, which are defined on $(\mathfrak{F} \times \mathfrak{F})$ and map into $\mathbb{R}$. These functions satisfy $|\delta(\cdot, \cdot)| = d(\cdot, \cdot)$, where $d(\cdot, \cdot)$ is a distance metric. $K(\cdot)$ is a kernel function, and $h_{K} = h_{K, n}$ is the smoothing parameter of the kernel $K(\cdot)$. To clarify, specific forms for $\delta(\cdot, \cdot)$ and $\beta(\cdot, \cdot)$ can be given. For example, if the functional data are 'smooth' curves, one might use the following family of locating functions:

where $\theta$ is a function that can be adapted to the data, $\mathcal{X}$ is a Hilbert space, and $\langle\cdot, \cdot\rangle$ denotes the inner product. Choosing $\beta(\cdot, \cdot)$ from such a family is motivated by its connection to the following minimization problem:

which can be viewed as a type of 'local polynomial' regression approach when considering a functional explanatory variable. Metrics, or more generally semi-metrics, based on derivatives could also be suitable for locating one curve relative to another. For example, one might define:

which is a semi-metric. This second family of locating functions is particularly well-suited for $\delta(\cdot, \cdot)$, as it measures the proximity between two elements of $\mathcal{X}$. Let $U$ be an open subset of a real Banach space $\mathfrak{F}$. If $f: U \rightarrow \mathbb{R}$ is differentiable $p+1$ times on $U$, it can be expanded using Taylor's formula:

with the following expressions for the remainder term $R_p(x)$ :

Here $a$ and $x$ must be points of $U$ such that the line segment between $a$ and $x$ lie inside $U, h$ is $x-a$, and the points $\xi$ and $\eta$ lie on the same line segment, strictly between $a$ and $x$. If we collect the equal mixed partials (assuming that they are continuous) then

where $J$ is a multi-index of $m$ components, and each component $J_i$ indicates how many times the derivative with respect to the $i$ th coordinate should be taken, and the exponent that the $i$th coordinate of $h$ should be raised to in the monomial $h^ J$. The multi-index $J$ runs through all combinations such that $J_1+\cdots +J_m = |J| = k$ in the sum. The notation $J!$ means $J_1!\cdots J_m!$. Let us specify our setting as in ^[11]. Let $U$ be an open subset of $\mathfrak{F}$. If $f: U \rightarrow \mathbb R$ is $p+1$ times differentiable on $U$, the Taylor expansion of $f$ around $x \in U$ is

where $D^j f(x)$ denote the $j$ th Fréchet derivative of $f$ at $x$, with $x$ and $y$ being points of $U$ and $d(\cdot, \cdot)$ the metric of $\mathfrak{F}$. The remainder is given by

as $d(x, y) \rightarrow 0$; see ^[26] and ^[90] for more details. Then, in our setting, by following a similar reasoning as in ^[9] and ^[11], the local polynomial estimator of order $p$ of the regression function denoted by $\widehat{r}(x)$ of $r(x)$ is defined as the first component of the estimate $\widehat{c}(x) = (\widehat{c}_{0}(x), \ldots, \widehat{c}_{p}(x))$ which is obtained by the following minimization problem

Furthermore, the estimator $\widehat{r}^{l}(x) = l! \, \, \widehat{c}_{l}(x)$, where $0\leq l \leq p$ can be expressed similarly as an estimator of the existing $l$th order derivatives of the regression function $r(\cdot)$. Then the estimator $\widehat{r}^{l}(x)$ can be written as

where $^{\top}\mathbf{Q}_{\beta}$ is the matrix defined by

where $^{\top}$ is the transpose symbol. Let us introduce

Keeping in mind this notation, we have the representation

Interestingly, this class of estimators includes both the classical Nadaraya-Watson estimator, which minimizes (2.3) when $p = 0$, and the local linear kernel estimator, which corresponds to $p = 1$.

3.   Main results for the regression operator

Let $x$ (resp. $y$) be a fixed point in $\mathfrak{F}$ (resp. $\in\mathbb{R}$), and $N_{x}$ (resp. $N_{y}$) denote a fixed neighborhood of $x$ (resp. $y$).

3.1.   Asymptotic distribution

In the remainder of the paper, we denote the closed ball in $\mathfrak{F}$ of center $x \in \mathfrak{F}$ and radius $h$ by

We define

where $l_{1}$ and $l_{2}$ are two real numbers. To establish the asymptotic behavior of the local polynomial of our estimator, we need some following assumptions.

(H1) For any $u > 0$, $\phi_x(u): = \phi_x(-u, u) > 0$ where $\lim\limits_{\substack{u \rightarrow 0}}\phi_x(u) = 0.$

It is easy to see that

When the function $|\delta(\cdot, \cdot)|$ satisfies the conditions of a metric or semi-metric, the expression $\varphi_x(u)$ can be understood as the probability of a ball in the set $\mathfrak{F}$ that is centered at $x$ and has a radius of $u$. When the value of $u$ approaches zero, the term "small ball probabilities" is frequently employed in the field of probability theory, which has been extensively studied (refer to ^[54] for a comprehensive summary of this topic in the context of Gaussian processes). The function $\varphi_x(\cdot)$, which is an extension of the small ball probability concept, serves a similar purpose in the functional situation as the density function does in the finite-dimensional context. In the context of multivariate nonparametric analysis, it is customary to estimate a specific quantity at a given position using a sufficiently large number of observations available in the vicinity. One commonly employed method for assuming this nature is to claim that the density function evaluated at this particular location possesses a strictly positive value. In the context of infinite dimensions, the absence of a reference measure, such as the Lebesgue measure in finite dimensions, necessitates the adoption of a similar assumption that does not rely on the concept of density. The objective of Hypothesis 1 (H1) is to incorporate the functional aspect, indicating that there are a sufficient number of observations surrounding $x$, hence justifying the estimation of the regression operator at the specific point $x$. The following conditions are needed in our analysis.

(H2) $(X_{i}, Y_{i})_{i\in\mathbb{N}}$ is an $\alpha$-mixing sequence and

(ⅰ) $\exists a > 0, \, \exists C > 0, \, \, \forall n\in\mathbb{N}, \, \, \alpha(n)\leq C\, n^{-a},$

(ⅱ) $\sup\limits_{i\neq j}\mathbb{P}((X_{i}, X_{j})\in B(x, h_{ K})\times B(x, h_{ K})) \leq\psi_{x}(h_{ K}),$ where $\psi_{x}(h_{ K})$ is such that there exists $\epsilon \in ]0, 1]$ for which $0 < \psi_{x}(h_{ K}) = O\left(\phi^{1+\epsilon}_{x}(h_{ K})\right);$

(H3) (ⅰ) $r(\cdot)$ and $\sigma(\cdot)$ are continuous in the neighbourhood of $x$, which means that $r(\cdot)$ and $\sigma(\cdot)$ are both in the set

and there exists $C > 0$, such that, almost surely,

(ⅱ) For any $k \in \lbrace 1, 2, \ldots, p, p+1 \rbrace,$ the quantities $\Psi^{(k)}(0)$ exist, where $\Psi^{(k)}(\cdot)$ denotes the ${k}th$ derivative of $\Psi(\cdot)$ defined by $\Psi(s) = \mathbb E(r(X)-r(x) \mid \beta(X, x) = s)$;

(H4) The locating operators $\beta(\cdot, \cdot)$ and $\delta(\cdot, \cdot)$ satisfy the following conditions:

(ⅰ) $\forall z \in\ \mathfrak{F}, \; C_1 |\delta(x, z)|\leq |\beta(x, z)|\leq C_2 | \delta(x, z)|;$

(ⅱ) $\sup\limits_{v\in B(x, r)} \mid \beta(v, x)- \delta(x, v) \mid = o(r)$;

(H5) The kernel $K(\cdot)$ is a bounded and positive function which is supported within $[-1, 1]$ and for which the first derivative $K^{\prime}(\cdot)$ satisfies: $K(1) > 0$, $K^{\prime}(t) < 0$, for $t\in[-1, 1];$

(H6) There exists a positive integer $n_0$ for which

and

where $d\mathbb P_{(X_{1}, X_{2})}$ is the cumulative distribution of $(X_1, X_2)$;

(H7) (ⅰ) For $a > 1$, the function $\phi_{x}(\cdot)$ satisfies

(ⅱ) The bandwidths $h_K$ satisfy

Before presenting our result, we need additional notation. In the sequel, we define certain quantities appearing in the dominant terms of the bias and of the variance in the asymptotic results. For $j \in \lbrace 1, 2 \rbrace$, let

where $\chi_{x}(\cdot)$ is a a function such that

For all $a > 0,$ and $b > 0$, let

3.2.   Comments on the conditions

Assumption (H2-ⅰ) requires that the mixing coefficients of the dependent case tend to zero at a suitably mild rate. Assumption (H2-ⅱ) describes the behavior of the joint distribution of the pair $(X_i, X_j)$ in relation to its marginal distributions, allowing us to present an explicit asymptotic variance term. Assumption (H3-ⅰ) imposes the usual moment condition on the responses and the covariance structure of the dependent sample, as detailed in ^[38]. Assumption (H3-ⅱ) specifies the necessary smoothness condition for the current setting. Assumption (H4) is a standard condition in nonparametric estimation. Condition (H5) is very usual in nonparametric estimation literature devoted to the functional data context. Notice that ^[70] symmetric kernel is not adequate in this context since the random process $d\left(x, X_i\right)$ is positive, therefore we consider $K(\cdot)$ with support $[0, 1]$. This is a natural generalization of the assumption usually made on the kernel in the multivariate case where $K(\cdot)$ is supposed to be a spherically symmetric density function. Assumption (H6) describes the behavior of the bandwidth $h$ in relation to the small ball probabilities and the kernel function $K(\cdot)$. Assumption (H7-ⅰ) is a technical condition illustrating the relationship between the small ball probability and an arithmetically $\alpha$-mixing coefficient, as discussed in ^[42]. Assumption (H7-ⅱ) is satisfied when $h_K = n^{-\varrho}$ and $\phi_x(h_k) = h_K^{\vartheta}$, provided $\vartheta > 0$ and $\beta < \varrho < 1/\vartheta$. In the following, we use the notation $Z \stackrel{D}{ = } \mathcal{N}(\mu, \sigma^{2})$ to indicate that the random variable $Z$ follows a normal distribution with mean $\mu$ and variance $\sigma^{2}$. The symbol ${\stackrel{D}{\rightarrow }}$ denotes convergence in distribution, and ${\stackrel{\mathbb{P}}{\rightarrow }}$ denotes convergence in probability.

Remark 3.1. According to ^[40], our methodology is heavily dependent on the distribution function $\phi(\cdot)$. This dependency is evident in our conditions and the convergence rates of our estimate (via the asymptotic behavior of the quantity $n\phi(h)$). More precisely, the behavior of $\phi(\cdot)$ around $0$ is of paramount importance. Consequently, the small ball probabilities of the underlying functional variable $X$ are crucial. In probability theory, the calculation of $\mathbb{P}(\|X - x\| < s)$ for "small" $s$ (i.e., for $s$ tending toward zero) and for a fixed $x$ is known as the "small ball problem". Unfortunately, there are solutions for very few random variables (or processes) $X$, even when $x = 0$. In several functional spaces, taking $x \neq 0$ presents formidable obstacles that may be insurmountable. Typically, authors emphasize Gaussian random variables. We refer you to ^[54] for a summary of the key findings regarding the probability of small balls. If $X$ is a Gaussian random element on the separable Banach space $\mathcal{E}$ and $x$ belongs to the reproducing kernel Hilbert space associated with $X$, then the following well-known result holds:

As far as we know, the results available in the published literature are essentially all of the form

where $\alpha, \beta, c_x$, and $C$ are positive constants, and $\|\cdot\|$ may be a supremum norm, an $L^p$ norm, or a Besov norm. The interested reader can refer to ^{[12,16,20,38,39,40,79]} for further discussion. Notably, the pioneering book by ^[38] extensively comments on the links between nonparametric functional statistics, small-ball probability theory, and the topological structure of the functional space $\mathcal{E}$.

In the following theorem, we present the limiting law.

Theorem 3.2. Assume that the assumptions (H1)–(H7) are satisfied. We have, as $n \rightarrow \infty,$

where

where $\widehat{c}_{p}(x)$ is a consistent estimator of $\Psi^{(p)}(0)$. Henceforth, we will denote $\widehat{c}_{p}(x)$ as $\widehat{\Psi}^{(p)}(0)$, and let $\widehat{r}(x) = \widehat{c}_{0}(x)$.

The proof of Theorem 3.2 is postponed until Section 6. To eliminate the bias term, we need the following additional assumption.

(H8) Assume that $\lim_{n\to \infty} n h_{K}^{2p+2} \phi_x(h_K) = 0$.

Corollary 3.3. Assume that the conditions (H1)–(H8) are satisfied. We have, as $n\rightarrow \infty$,

Remark 3.4. To cancel the bias term, we need $n h_{K}^{2p+2} \phi_x(h_K) \rightarrow 0$, as $n \rightarrow \infty$. Consequently, the last condition and condition $n \phi_x(h_K) \rightarrow \infty$ are satisfied as soon as $h_K = n^{-\xi}$, $0 < \xi < 1$, with and $\phi_x(h_K) = h_K^c$, for $\frac{1}{\xi }-(2p-2) < c < \frac{1}{\xi }$.

3.3.   Application to the confidence regions

A lower $\alpha$ th quantile of the vector $V_n$ is any quantity $\tau_{n \alpha} \in \mathbb{R}^{p+1}$ satisfying $\tau_{n \alpha} = \inf \left\{\varepsilon: \mathbb P\left(V_n \leq \varepsilon\right) \geq \alpha\right\}$, where $\varepsilon$ is an infimum over the given set only if there does not exist a $\varepsilon_1 < \varepsilon$ in $\mathbb{R}^{p+1}$ such that $\mathbb P\left(V_n \leq \varepsilon_1\right) \geq \alpha$. We can, without loss of generality, assume $\mathbb P\left(V_n \leq \tau_{n \alpha}\right) = \alpha$. Since variance matrix $\Sigma(x) = \sigma^{2}(x) S^{-1} V S^{-1}$ is unknown and assumed non-singular, see Corollary 4 of Appendix 1 of ^[4], there exists a non-singular matrix $C(x)$ such that

Let us first give a consistent estimate of $\Sigma(x)$. Making use of the decomposition of $\chi_x(t)$ in (3.1), one may estimate $\chi_x(t)$ by

where

Hence we have the following estimates, for $j = 1, 2,$

In a similar way, for all $a > 0,$ and $b > 0$, we have

Hence one can consistently estimate $\Sigma(x)$ by $\widehat\Sigma(x) = \widehat\sigma^{2}(x) \widehat S^{-1}\widehat V \widehat S^{-1},$ where

For $n$ large enough, we have

An application of Slutsky with Corollary 3.3 gives the following result.

Corollary 3.5. Assume that the conditions (H1)–(H8) are satisfied. We have, as $n\rightarrow \infty$,

This corollary can be applied directly to the construction of the confidence regions.

3.4.   Uniform almost-complete convergence

In this part, we will establish the rate of uniform almost complete convergence over some subset of $\mathcal{F}$. The following definition is needed.

Definition 3.6. Let $\epsilon > 0$ be given number. A finite set of points $x_{1}, x_{2}, \ldots, x_{N}$ in $\mathcal{F}$ is called an $\epsilon -$net for S if $S \subset \cup_{k = 1}^{N} B(x_{k}, \epsilon).$ The quantity $\Psi_{s}(\epsilon) = \log(N_{\epsilon}(S)),$ where $\Psi_{s}(\epsilon)$ is the minimal number of open balls in $\mathcal{F}$ of radius $\epsilon$ which is necessary to cover $S$, is called Kolmogorov's $\epsilon-$entropy of the set $S$.

The concept of entropy, introduced by Kolmogorov in the mid-1950s (see ^[52]), serves as a measure of set complexity, indicating that high entropy implies a significant amount of information is required to accurately describe an element within a given tolerance $\varepsilon$. Consequently, the selection of the topological structure, specifically the choice of the semi-metric, plays a crucial role when examining asymptotic results that are uniform over a subset $S$ of $\mathcal{F}$. In particular, we subsequently observe that a well-chosen semi-metric can enhance the concentration of the probability measure for the functional variable $X$, while minimizing the $\varepsilon$-entropy of the subset $S$. Ferraty and Vieu ^[38] emphasized this phenomenon of concentration of the probability measure for the functional variable by calculating small ball probabilities in different standard scenarios (refer to ^[41]). For readers interested in these concepts (entropy and small ball probabilities) and/or the utilization of Kolmogorov's $\varepsilon$-entropy in dimensionality reduction problems, we recommend referring to ^[53] or/and ^[69], respectively. We establish the uniform almost-complete convergence of our estimator on some subset $S_{\mathcal{F}}$ such that:

where $x_{k} \in \mathcal{F}$ and $\iota_{n}$ (resp. $d_{n}$) is a sequence of positive real numbers. To establish our result, we need the following hypotheses.

(U1) The hypothesis (H1) satisfies the following condition: there exists a differentiable function $\phi(\cdot)$, such that

and there exists $n_0$ such that for each $n < n_0$

where $\phi^{\prime}(\cdot)$ represents the first derivative of $\phi(\cdot)$ and $\phi(0) = 0$;

(U2) The hypothesis (H2) is satisfied uniformly on $x \in S_{\mathcal{F}};$

(U3) The function $\beta(\cdot, \cdot)$ satisfies (H4) and the following Lipschitz condition, there exists a positive constant $C$, for all $x_1, x_2$ in $\mathcal{F}$, we have

(U4) The regression operator $r(\cdot)$ satisfies: there exists a positive constant $C$ and $s > 0$, such for each $x\in \mathcal{F}$ and $z\in B(x, h_{K})$, we have

(U5) The kernel function $K(\cdot)$ satisfies (H5) and is Lipschitzian on $[-1, 1]$;

(U6) In addition to (H7), the Kolmogorov's $\varepsilon$-entropy of $S_{\mathcal{F}}$ satisfies:

(ⅰ) $\exists n_0, \, \forall n > n_0, \frac{\log ^2 n}{n \phi(h)} < \psi_{S_{\mathcal{F}}}\left(\frac{\log n}{n}\right) < \frac{n \phi(h)}{\log n}$.

(ⅱ) $\exists \lambda > 1$, such that

3.5.   Comments on the conditions

Conditions (U1) and (U2) correspondingly serve as the uniform counterparts of (H1) and (H2). Assumption (U3), initially introduced and discussed by ^[9], plays a pivotal role in our methodology, particularly when computing the leading dominant terms in our asymptotic results. Moreover, (U4) proves essential for evaluating the bias component of the uniform convergence rates, and (U5) is a classical requirement in functional estimation within finite or infinite-dimensional spaces, with examples of kernel functions satisfying this condition available in ^[38]. The final condition regarding entropy, (U6), implies that $\psi_{S_{\mathcal{F}}}\left(\frac{\log n}{n}\right) = o(n\phi(h_{K}))$ as $n$ tends to infinity. However, in certain "usual" scenarios, such as a Hilbert space with a projection semi-metric, one can consider $\Psi_{si_{\mathcal{F}}}\left(\frac{\log n}{n}\right) \sim C \log n$, and (H6 i) is fulfilled whenever $(\log n)^{2} = O(n \phi(h))$. For further insights, one may refer to Example 4 in ^[41].

We are now equipped to state the main result of this section concerning the uniform almost complete convergence with rate.

Theorem 3.7. Assume that the assumptions (U1)–(U6) are satisfied. We have as $n\rightarrow \infty$,

^‡Let $\left(u_{n}\right)$ for $n \in \mathbb{N}$ be a sequence of real random variables. We say that $\left(u_{n}\right)$ converges almost-completely (a.co.) toward zero if, and only if, for all $\epsilon > 0$, $\sum_{n = 1}^{\infty} \mathbb{P}\left(\left|u_{n}\right| > \epsilon\right) < \infty$. Moreover, we say that the rate of almost-complete convergence of $\left(u_{n}\right)$ toward zero is of order $v_{n}$ (with $v_{n} \rightarrow 0$), and we write $u_{n} = O_{a.co.}\left(v_{n}\right)$ if, and only if, there exists $\epsilon > 0$ such that $\sum_{n = 1}^{\infty} \mathbb{P}\left(\left|u_{n}\right| > \epsilon v_{n}\right) < \infty$. This type of convergence implies both almost sure convergence and convergence in probability.

where

4.   The conditional distribution function

In this section, we will establish results analogous to Theorem 3.2 for the local polynomial estimator of order $p$ of the Cumulative distribution function which we will denote by $\widehat{F}^{x}(\cdot)$. We notice that the distribution function is expressed in terms of regression depending on the choice of the function $\psi(Y)$ (see Eq (2.1)). Then if $\psi_y(Y) = \mathbf{1}_{Y \leq y}$, the distribution function is defined by

The functional local polynomial estimator of $F^{x}(y)$ is based on the minimization, with respect to $\widehat{a} = (\widehat{a}_{0}, \ldots, \widehat{a}_{p}),$ of

where $L(\cdot)$ is cumulative distribution function and ($h_{L} = h_{L, n}$) is a sequence of positive real numbers. In the realm of functional statistics, the estimation of the conditional cumulative distribution function holds significant importance both theoretically and practically. This is evident in various applications, such as reliability and survival analysis. Additionally, within the domain of nonparametric statistics, numerous prediction tools, including conditional density, conditional quantiles, and conditional mode, play crucial roles. It is worth noting that these considerations apply particularly when $p$, is equal to 1. The almost-complete convergence of the estimator $\widehat{F}^{x}(\cdot)$ in the case of $p = 1$ has been investigated by ^[31]. Furthermore, ^[14] has explored the asymptotic normality of the local linear estimation of the conditional function in the scenario of independent observations. For the development of our estimator, we adopt a similar approach as used for the regression operator. Consequently, the local polynomial estimator of $\widehat{F}^{x}(y)$ can be explicitly expressed as follows:

where

We denote

4.1.   Main results

To establish the asymptotic convergence of $\widehat{F}^{x}(y)$ we need some notation and assumptions. For any $l\in \{0, 2, \ldots, 2p\}$, we set

(M1) For any $k\in \{1, \ldots, p+1\}$, the quantities $\psi_{l}^{(k)}(0)$ exist, where $\psi_{l}^{(k)}(\cdot)$ denotes the $k$th order derivative of $\psi_{l}(\cdot)$.

(M2) The kernel $K(\cdot)$ satisfies the assumption (H5) and its first derivative $K^{(1)}(\cdot)$ satisfies:

(M3) The kernel function $L(\cdot)$ is a positive function, symmetric, bounded, integrable, and such that and

(M4) The coefficients of $\alpha$-mixing sequence$(X_{i}, Y_{i})_{i\in\mathbb{N}}$ satisfies (H2) and the following condition,

(M5) The function $\phi_{x}(\cdot)$ satisfies the assumption (H1).

(M6) Let $(w_{n})$ and $(q_{n})$ be sequences of positive integers tending to infinity, such that $(w_{n} +q_{n}) \leq n$, and

(ⅰ) $q_{n} = o\left((n\, \phi_{x}(h_{ K}))^{\frac{1}{2}}\right)$ and $\lim\limits_{\substack{n \rightarrow +\infty}}\left(\frac{n}{\phi_{x}(h_{ K})}\right)^{\frac{1}{2}}\alpha(q_{n}) = 0,$

(ⅱ) $r_{n}q_{n} = o\left((n\phi_{x}(h_{ K}))^{\frac{1}{2}}\right)$ and $\lim\limits_{\substack{n \rightarrow +\infty}}r_{n}\left(\frac{n}{\phi_{x}(h_{ K})}\right)^{\frac{1}{2}}\alpha (q_{n}) = 0,$ where $r_{n}$ is the largest integer, such that $r_{ n} (w_{n} +q_{n}) = O(n)$.

Theorem 4.1. Under assumptions (M1)–(M6), we have

where

Remark 4.2. Let $x \in \mathcal{F}$ be a fixed element, and $y \in \mathbb{R}$. If we define $\psi_y(Y) = \mathbf{1}_{]-\infty, y]}(Y)$, then the operator $r_\psi(x) = \mathbb{E}(\psi_y(Y) \mid X = x)$ represents the conditional cumulative distribution function (CDF) of $Y$ given $X = x$, denoted as $F(y \mid x) = \mathbb{P}(Y \leq y \mid X = x)$. This can be estimated as $\tilde{F}(y \mid x) : = \widehat{r}_{\psi, n}(x)$. For a given $\alpha \in (0, 1)$, the $\alpha$-th order conditional quantile of the distribution of $Y$ given $X = x$ is defined as $q_\alpha(x) = \inf \{ y \in \mathbb{R} : F(y \mid x) \geq \alpha \}$. If $F(\cdot \mid x)$ is strictly increasing and continuous in a neighborhood of $q_\alpha(x)$, then $F(\cdot \mid x)$ has a unique quantile of order $\alpha$ at $q_\alpha(x)$, i.e., $F(q_\alpha(x) \mid x) = \alpha$. In such cases:

which is uniquely estimated by $\widehat{q}_{n, \alpha}(x) = \tilde{F}^{-1}(\alpha \mid x)$. Conditional quantiles have been extensively studied when the predictor $X$ is of finite dimension. Since $F(q_\alpha(x) \mid x) = \alpha = \tilde{F}(\widehat{q}_{n, \alpha}(x) \mid x)$, and $\tilde{F}(\cdot \mid x)$ is continuous and strictly increasing, we have, for any $\epsilon > 0$, there exists $\eta(\epsilon) > 0$ such that for all $y$:

This implies that, for any $\epsilon > 0$, there exists $\eta(\epsilon) > 0$ such that:

This result may be immediately used to establish the consistency of the quantile estimator. This topic will be investigated in future research.

Remark 4.3. Over time, it has become increasingly clear that the performance of kernel estimates deteriorates as the dimensionality of the problem increases. This decline is primarily due to the fact that, in high-dimensional spaces, local neighborhoods often lack sufficient sample observations unless the sample size is extraordinarily large. Consequently, in kernel estimation, computing local averages is not feasible unless the bandwidth is significantly wide. This overarching issue is known as the curse of dimensionality ^[10]. The paper by ^[77] provides a comprehensive analysis of the feasibility and challenges of high-dimensional estimation, complete with examples and computations. For recent references, see ^[6,15,16,35]. To address the difficulties associated with high-dimensional kernel estimation and simplify the process, a wide range of dimension reduction techniques have been developed. These techniques major to explain the main features of a set of sample curves using a small set of uncorrelated variables. One effective solution is to generalize principal component analysis (PCA) to the context of a continuous-time stochastic process ^[33]. The asymptotic properties of the estimators of Functional Principal Component Analysis (FPCA) have been extensively studied in the general context of functional variables ^[30]. Nonparametric methods have been developed to perform FPCA for cases involving a small number of irregularly spaced observations of each sample curve ^[50,88]. As in the multivariate case, interpreting principal component scores and loadings is a valuable tool for uncovering relationships among the variables in a functional data set. To avoid misinterpretation of PCA results, a new type of plot, named Structural and Variance Information plots, was recently introduced by ^[25]. Numerical results suggest that regularization using a smoothness measure often yields more satisfactory outcomes than the FPCA approach. The FPCA method assumes that the top eigenfunctions of the predictors, which are unrelated to the coefficient function or the response, provide a good representation of the coefficient function. While it is generally believed that neither approach will consistently outperform the other, the Reproducing Kernel Hilbert Space (RKHS) method is an interesting alternative that merits attention ^[58,89]. Spline smoothing is one of the most popular and powerful techniques in nonparametric regression ^{[36,46,47,84]}. Penalized splines have gained widespread use in recent years due to their computational simplicity and their connection to mixed effects models ^[86]. These methods have also found significant success in Functional Data Analysis (FDA), as evidenced by tools like the R package refund. However, there is a consensus that the asymptotic theory of penalized splines is difficult to derive, and many theoretical gaps remain, even in the standard nonparametric regression setting. To our knowledge, there are very few theoretical studies on penalized splines for FDA ^[87]. Most approaches to functional regression are based on minimizing some $L_2$-norm and are therefore sensitive to outliers. Finally, we can mention methods based on the delta sequences ^[23] and the wavelet methods ^[34].

5.   Concluding remarks

Local polynomial fitting exhibits various statistically significant characteristics, particularly in the intricate domain of multivariate analysis. As functional data analysis gains traction in the field of data science, there arises a need for a specialized theory focused on local polynomial fitting. This study employs local higher-order polynomial fitting to address the complex challenge of estimating the regression function operator and its partial derivatives for stationary mixing random processes, represented as $(Y_i, X_i)$. For the first time, we have achieved significant progress by demonstrating the joint asymptotic normality of the estimates for the regression function and its partial derivatives, particularly applicable to strongly mixed processes. Additionally, we derive precise formulas for the bias and the variance-covariance matrix of the asymptotic distribution. We establish the uniform strong consistency of the regression function and its partial derivatives across compact subsets, providing a detailed analysis of their convergence rates. Importantly, these conclusions are grounded in general conditions that form the foundation of the underlying models. To illustrate practical utility, we utilize our findings to compute confidence regions for individual points. Furthermore, we extend our concepts to encompass the nonparametric conditional distribution and acquire its limiting distribution. There are multiple avenues for further methodological development. As a prospective direction, relaxing the stationarity assumption and exploring similar uniform limit theorems for local stationary functional ergodic processes would be fruitful. Additionally, considering the functional $k$NN local polynomial approach and expectile regression estimators in future investigations could yield alternative estimators benefiting from the advantages of both methods, as discussed in ^[20] and referenced in ^[19]. Investigating the extension of our framework to the censored data setting would also be of interest. In an upcoming investigation, obtaining a precise uniform-in-bandwidth limit law for the proposed estimators will be essential. This outcome would allow the bandwidth to vary across a comprehensive range, ensuring the estimator's consistency and serving as a valuable practical guideline for bandwidth selection in nonparametric functional data analysis. It is crucial to acknowledge that advancing research in this direction poses a more challenging task compared to previous efforts. The primary challenge lies in the need to develop new probabilistic results, such as inequalities and maximal moment inequalities specifically tailored for independent and identically distributed (i.i.d.) samples, as discussed in ^[81].

6.   Proof of the main result

This section is devoted to the proof of our main result. The previously presented notation continues to be used in the following. Before studying the limit law of $\widehat{c}(x)$, we need to center the vector $V_{n}$ as follows. Let

where $V_{n}^{\ast} = ^{\top}(u_{n, 0}^{\ast}, \ldots, u_{n, p}^{\ast}).$ Making use of assumption (H4) and by the Taylor expansion, we obtain

Therefore, we have

Making used of Eq (6.1), we have:

The rest of the proof for this theorem relies on the following lemmas, the proofs of which are provided in the appendix.

Lemma 6.1. Under the assumptions (H1)–(H7), we have, as $n\rightarrow \infty$,

Hence, considering Lemma 6.1, Theorem 3.2 will hold if we can furnish the proof.

Lemma 6.2. Under the assumptions of Theorem 3.2, we have, as $n\rightarrow \infty$,

Author contributions

Oussama Bouannani: Conceptualization, formal analysis, investigation, methodology, validation, writing – original draft, review, edition; Salim Bouzebda: Conceptualization, formal analysis, methodology, validation, writing – original draft, review, edition. All authors have read and approved the final version of the manuscript for publication.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors extend their sincere gratitude to the Editor-in-Chief, the Associate Editor, and the three referees for their invaluable feedback. Their insightful comments have greatly refined and focused the original work, resulting in a markedly improved presentation.

Conflict of interest

The authors declare no conflict of interest. Salim Bouzebda is the(a) Guest Editor of special issue "Advances in Statistical Inference and Stochastic Processes: Theory and Applications" for AIMS Mathematics. Salim Bouzebda was not involved in the editorial review and the decision to publish this article.

A.   Appendix

Lemma A.1. (See ^[92]) Under assumptions (H1), (H2-ii) and (H5)–(H7-i), we have

(1) $\mathbb E(K_{1}^{j}) = M_{j}\phi_{x}(h_{K}),$ for $j = 1, 2$;

(2) $\mathbb E(K_{1}^{a}\beta_{1}) = o(h_{K}\phi_{x}(h_{K})),$ for all $a > 0;$

(3) $\mathbb E(K_{1}^{a}\beta_{1}^{b}) = N(a, b)h_{K}^{b}\phi_{x}(h_{K})+o(h_{K}^{b} \phi_{x}(h_{K})),$ for all $a > 0$ and $b = 2, 4$;

(4) For all $(k, l) \in \mathbb{N}^{\ast} \times \mathbb{N}, \; \mathbb E\left(K^{k}_{1} \; |\beta_{1} |^{l}\right) \leq C\; h_{K}^{l} \phi_{x}(h_{K});$

(5) For all $(a, c, l, s) \in \mathbb{N}^{\ast} \times \mathbb{N}^{\ast} \times \mathbb{N}\times \mathbb{N}, \; \mathbb E\left(K^{a}_{1} K^{c}_{2} \; |\beta_{1} |^{l} |\beta_{2} |^{s} \right) \leq C\; h_{K}^{l+s} \psi_{x}(h_{ K}).$

Proof of Lemma 6.1. To prove this lemma, we use the sufficient convergence condition in probability. For this, it suffices to prove the following formulas

and

Using Lemma A.1, we get

Concerning Eq (A.1), we can write

For the term $Q_{n1}$, we have

Moreover, for computing the term of $Q_{n2}$, we used the same technique as in ^[62], we define the sets $W_{1}$ and $W_{2}$ as follows

and

where the sequence $u_{n}$ is selected in such a way that $u_{n}\to +\infty$ as $n\to +\infty.$ Based on the aforementioned splitting, we obtain

and

For the sum $Q_{1, n}$, by assumptions (H2-ii), (H5) and by Lemma of ^[92], we have

Then, we readily infer

On the other hand, for computing the sum of covariance $Q_{1, n}$, we use the inequality for the bounded mixing processes, then for any $l \neq i$, we have

Next, using the inequality

we derive

Therefore, we readily obtain

Subsequently, by choosing $u_{n} = \lfloor\phi^{1+\epsilon}_{x}(h_{ K}))^{-1/a}\rfloor$ and by assumption (H7), we obtain

Hence the proof is complete. □

Proof of Lemma 6.2. We consider a given vector of real numbers, denoted as $\mathbf{a} = ^{\top}\left(\alpha _{0} \ldots \alpha _{p}\right)\neq 0$, then

where

By the Cramér-Wold theorem and Slutsky's theorem, to show (13.6), it suffices to prove the following two claims:

and

First, we proof Claim (A.7). We have

Next, by the same arguments as those used by ^[72] and ^[40], we get

and

then, using Lemma A.1, we obtain

Now, we prove Claim (A.8). Using The CLT by ^[56] (see Corollary 2.2, Page 196), which rests on the asymptotic behavior of the quantity

where $\Delta_{i} = \frac{1}{\sqrt{n}} \left(Z_{i}-\mathbb E(Z_{i}) \right)$ in addition to the assumptions:

Assumption 1. (A1) There exists a sequence $\tau_n = o(\sqrt{n})$, such that

(i) $\tau_n = o(\sqrt{n})$ such that $\tau_n\leq \left(\max_{i = 1, \ldots, n} C_{i} \right)^{-1},$ where $C_{i} = {\rm ess} \sup\limits_{\omega \in \Omega} |\Delta_{i}|,$

(ii) for all $\epsilon > 0$, $\frac{n}{\tau_n}\ \alpha(\epsilon \tau_n) \to 0$, for all $\epsilon > 0.$

Assumption 2. (A2) There exists a sequence $(m_n)$ of positive integers tending to $\infty$, such that

(i) $n m_n \gamma_n = o(1),$ where $\gamma_n: = \max\limits_{1\leq i\not = j\leq n}\left(\mathbb E|\Delta_{i} \Delta_{j}| \right),$

(ii) for all $\epsilon > 0$, $\frac{n}{\tau_n} \alpha(\epsilon \tau_n)\to 0.$

We start by computing the limit of (A.10). In order to do so, let us observe that

Using the fact that

in combination with Lemma A.1, we readily find

and

Under assumptions (H1), we have

Subsequently, the assertion (A.7) suggests that $\mathbb{E}(Z_{1})\rightarrow 0$ as $n \rightarrow \infty$. Consequently, we infer that $A_{2}\rightarrow 0$ as $n \rightarrow \infty$. Concerning Assumption (1), the fact that the kernel $K(\cdot)$ is bounded, allows us to infer that

Therefore, an appropriate choice is the following

Furthermore, this choice gives, for all $\epsilon > 0$

Concerning Assumption (2), using assumptions (H2-ⅱ) and (H3), we have

Elsewhere, using the fact that

we obtain

thus,

We select

such that $\lfloor\cdot\rfloor$ represents the function of the integer part. It is evident that, under assumption (H7), $m_n\to\infty$. Furthermore, by substituting the expression for $m_n$, we readily get

Once more, considering assumption (H2-ⅰ) and (H7), we have

Hence the proof is complete. □

Proof of Theorem 3.7. According to the formula of (6.2), we deduce the following error formula

where $e_{n}(x) = ^{\top}(u_{n, p+1}(x), \ldots, u_{n, 2p+1}(x))$. Hence, Theorem 3.7 is a direct result of the following lemmas.

Lemma A.2. Assume that the conditions (U1), (U3), (U5) and (U6) are fulfilled. We have, as $n\rightarrow \infty,$

Lemma A.3. Assume that the conditions (U1)–(U6) are fulfilled. We have, as $n\rightarrow \infty,$

Lemma A.4. Assume that the conditions (U1)–(U5) are fulfilled. We have, as $n\rightarrow \infty,$

□

Proof of Lemma A.2. The proof of this lemma relies on Proposition A.11-ⅰ in ^[38]. In order to apply the latter, we need to compute a certain quantity. In a first attempt, we evaluate the expression $\mathbb E(u_{nj}(x))$. For this, we use Lemma A.1 and by the fact that observations are identically distributed, we have

Moreover, for the covariance term, we have :

where

Under some of the dependence assumptions, we treat the term of $S_{n, \alpha, l}^{2}(x)$. Then we have

with

and

For the term variance using Lemma A.1 and we deduce that

Concerning the covariance term $R_{n, 1}(x)$, we use the same idea as those used in the Eq (A.3) with the choice $u_{n} = \phi^{-\epsilon}_{x}(h_{ K})$, we get

and for the term $R_{n, 1}(x)$ we applied Rio inequality, see Theorem B.3 (ⅱ). So for this technique, we need to compute the absolute moments of the random variable $A_{j}^{l}(x)$. Then, using a Newton expansion, we have

where $C^{i}_{m} = \frac{m!}{i!(m-i)!}.$ Next, by applying Lemma A.1, we get

By following the same reasoning used to establish Eq (A.4) with the choice $u_{n} = \phi^{-\epsilon}_{x}(h_{ K})$ permits to get:

Finlay, by Eqs (A.16), (A.17) and (A.18), we deduce that

On the other hand, we can write

where $A_{i}^{l}(x)$ is defined in (A.15). Now, we apply the Definition 3.6 to show the uniform convergence of our estimator on a subset $S$ of $\mathcal{F}$. For this aim, we take

Let us consider the following decomposition:

Concening the term $T_{2}$, for all $\epsilon > 0$, we have

Next, we can apply the Proposition A.11 of ^[38] for any $m > 2, \, \tau > 0, \, \varpi \geq 1$ and for certain $0 < C < \infty$,

where

By using equation of (A.19) and under assumption (U1), we obtain that

Now, for $\eta > 0$, we choose

Therefore, by utilizing the fact that $\Psi_{S _{ \mathcal{F}}}(\epsilon) = \log(d_{n})$ and by taking C $\eta^{2} = \lambda$, we obtain under the assumption (U6) the following terms

Finally, for $\eta$ large enough, we infer

Using the fact that

we get

For the term $T_1$ we have

Evaluation of the term $L_{1}^{j}$. Under assumption (U3) we have

which gives the following expression

Making use of the Lipschitz condition on $K(\cdot)$ and by hypotheses (U3) enables us to directly write

Therefore, we readily obtain

Under assumption (U5) en by combined the last inequality with (A.22) implies that

where

Using a similar approach to the one employed in the proof of (13.14), under the assumptions (U1), (U2) and (U6), we obtain

Next, by employing a similar proof to the one applied in the evaluation of $T_{2}$, we infer that

Concerning the term of $T_{3}$, it is obvious that

we infer that

This completes the proof. □

Proof of Lemma A.4. By conditioning on $X_{1}$, we have

so under assumption (U4) and by using the uniform version of Lemma A.1, we obtain

Concerning the term $\sup\limits_{x\in S_{\mathcal{F}}} |\mathbb E(e_{nj}(x))| = O(1)$ is already treated in Eq (A.12). □

Proof of Theorem 4.1. Using similar reasoning employed for the regression function, we show that

where

and

For any given vector of real numbers $\mathbf{a} = ^{\top}(\alpha _{0}, \ldots, \alpha _{p})\neq 0$, we have

where

According to the Cramér-Wold theorem and Eq (A.24), Eq (A.23), will be verified if we prove the following two statements:

□

Proof of statement A.25. We utilize Bernstein's big-block method, as employed in Theorem 3.1 of ^[55]. The set $(1, 2, \ldots, n)$ is partitioned into $2\upsilon_{n} + 1$ subsets, each containing large blocks of size $(w_{n})$ and small blocks of size $(q_{n})$, by considering

Assumption (M6) allows us to define the size of the large block as follows

Afterward, under the same assumptions and employing straightforward calculations, we obtain:

It can be easily inferred that, as $n$ tends to infinity,

Presently, we partition the sum $\sum\limits_{i = 1}^{n} (R_{i}-\mathbb E(R_{i}))$ into distinct large and small blocks as outlined below. Let

One can see that

where the random variables $F_{j}, F^{'}_{j}$ and $F_{n}$ are defined by

Subsequently, making use of Slutsky's theorem, we establish that the two terms $\frac{1}{\sqrt{n}} S_{2, n}$ and $\frac{1}{\sqrt{n}} S_{3, n}$ converge to zero in probability. Before applying Lindeberg-Feller conditions to establish the asymptotic normality of the term $\frac{1}{\sqrt{n}}S_{1, n}$, it is imperative to demonstrate that the variables $F_{j}$ are asymptotically independent. Therefore, we obtain

Using second-order stationarity, we get

Therefore

To derive the covariance term, we adopt the identical procedures outlined in the verification of Eq (A.2). Then we set

and

Based on the aforementioned splitting, we obtain

Utilizing the inequality

stated in Lemma A.1 and given the conditions (H2-ⅱ) and (H4-ⅱ), we obtain the following

Then, we readily infer

In relation to the summation over the set $G_{2}$, the utilization of Theorem B.3 (ⅱ) gives

Conditioned on $X_{i}$, and using (A.30) along with assumption (M3), we obtain

We readily infer that

Using the fact that $\frac{\psi_{x} (h_{ K}) }{\phi^{2}_{x}(h_{ K})}$ is bounded, we select $c_{n} = \left\lfloor \phi_{x}(h_{ K})\right\rfloor^{-\frac{1}{p\delta}}$ and by assumptions (M4), and (M5) we obtain

Furthermore, for the term $A_{2}$, we have

Consequently by (A.27), we get

Let's now examine the term of $S_{2, n}^{2}$

Besides, from (A.27) and (A.28), we get $\frac{\upsilon w}{n}\longrightarrow 1$. Then, we obtain

In order to demonstrate the asymptotic independence of the variables $F_{j}$, we employ Lemma B.1. By setting $V_{j} = \exp\left(\frac{i tF_{j}}{\sqrt{n}}\right)$, the ensuing relationship

Therefore, according to formula (A.33), the term variance $S_{1, n}$ is computed as follows:

where

Using Lemma 6.3 of ^[14], we obtain

Then, by using the fact that $\frac{\upsilon w}{n}\longrightarrow 1$ and under Lemma A.1, we get

Finally, our attention turns to Lindeberg's central limit theorem concerning $F_{j}$. It is then enough to demonstrate that for any $\epsilon > 0,$

Under assumption (M2), we have

According to (A.27), we obtain

Hence, for any given $\epsilon$ and when $n$ is sufficiently large, the set $\lbrace|F_{j}| > \epsilon\sqrt{n}V_{F}(x, y)\rbrace$ becomes empty. Consequently, the demonstration of (A.25) is now concluded. □

Proof of statement A.26. We have

Next, we compute the term $N(x, y)$. Notice that

Making use of the Taylor expansion under the assumptions (M2) and (M3), we obtain

Therefore, we readily obtain

Observe that $\psi_{k}(0) = 0$, we get

By combining Eqs (A.34) and (A.35), we can derive

So the statement (A.26) is proved. □

B.   Appendix

This appendix contains supplementary information that is an essential part of providing a more comprehensive understanding of the paper.

Lemma B.1. ^[82] Let $V_{1}, \ldots, V_{L}$ be random variables measurable with respect to the $\sigma$-algebras $\mathscr{F}_{i_{1}}^{j_{1}}, \ldots, \mathscr{F}_{i_{L}}^{j_{L}}$ respectively with $1 \leqslant i_{1} < j_{1} < i_{2} < \cdots < j_{L} \leqslant n, \quad i_{l+1}-j_{l} \geqslant w \geqslant 1$ and $\left|V_{j}\right| \leqslant 1$ for $j = 1, \ldots$, L. Then

where $\alpha(w)$ is the strongly mixing coefficient.

Theorem B.2. (Lindeberg central limit theorem). For each $n \geq 1$, let $\left\{U_{n 1}, \ldots, U_{n r_n}\right\}$ be a collection of independent random variables such that $\mathbb{E} (U_{n j}) = 0$ and $\operatorname{Var}(U_{n j}) < \infty$ for $j = 1, \ldots, r_n$.

Then

if for every $\epsilon > 0$

The following theorem is Proposition A.10. in ^[38].

Theorem B.3. Assume that $\left(T_n\right)_{n \in \mathbb{Z}}$ is $\alpha$-mixing. Let us, for some $k \in \mathbb{Z}$, consider a real variable $\mathcal{T}$ (resp. $\mathcal{T}^{\prime}$) which is $\mathcal{A}_{-\infty}^k$-measurable (resp. $\mathcal{A}_{n+k}^{+\infty}$-measurable).

(i) If $\mathcal{T}$ and $\mathcal{T}^{\prime}$ are bounded, then:

(ii) If, for some positive numbers $p, q, r$ such that $p^{-1}+q^{-1}+r^{-1} = 1$, we have $E \mathcal{T}^p < \infty$ and $E \mathcal{T}^{\prime q} < \infty$, then: