Uniform Consistency for Functional Conditional U-Statistics Using Delta-Sequences

Abstract

U-statistics are a fundamental class of statistics derived from modeling quantities of interest characterized by responses from multiple subjects. U-statistics make generalizations the empirical mean of a random variable X to the sum of all k-tuples of X observations. This paper examines a setting for nonparametric statistical curve estimation based on an infinite-dimensional covariate, including Stute’s estimator as a special case. In this functional context, the class of “delta sequence estimators” is defined and discussed. The orthogonal series method and the histogram method are both included in this class. We achieve almost complete uniform convergence with the rates of these estimators under certain broad conditions. Moreover, in the same context, we show the uniform almost-complete convergence for the nonparametric inverse probability of censoring weighted (I.P.C.W.) estimators of the regression function under random censorship, which is of its own interest. Among the potential applications are discrimination problems, metric learning and the time series prediction from the continuous set of past values.

1. Introduction

The regression problem has been studied by statisticians and probability theorists for many years to keep up with the various problems and topics brought up by technological and computational advances, resulting in the creation of many advanced and complex techniques. Among the problems addressed are modeling, estimation method applications, and tests. In this paper, we are interested in nonparametric regression estimation. Unlike the parametric framework, where one must estimate a finite number of parameters based on a specified structural model a priori, nonparametric estimation does not require any specific structure; instead, it allows the data to speak for themselves. However, as natural drawbacks, nonparametric procedures are more susceptible to estimation biases and losses in convergence rates than parametric methods. Since their introduction, kernel nonparametric function estimation approaches have garnered a significant amount of attention; for references to research literature and statistical applications in this area, consult [1,2,3,4,5,6,7] and the references therein. Popular as they may be, they represent only one of the numerous methods for developing accurate function estimators. These include nearest-neighbor, spline, neural network, and wavelet approaches. In addition, these techniques have been applied to a vast array of data types. This article will focus on constructing consistent estimators for the conditional U-statistics for functional data based on the delta sequence. The theory of U-statistics and U-processes, which was initially introduced in the seminal work of [8], has garnered a significant amount of interest over the course of the last few decades as a result of the diverse applications to which it has been applied. U-statistics can be utilized to solve complex statistical problems. Among the examples are nonparametric regression, density estimation, and goodness-of-fit tests. Furthermore, U-statistics contribute to the study of estimators with various degrees of smoothness (including function estimators). [9], for instance, analyzes the product limit estimator for truncated data applying a.s. uniform bounds for $\mathbb{P}$-canonical U-processes. [10] introduces two new normality tests based on U-processes. Using the findings of [11,12,13] provided new normality tests that used as test statistics weighted $L_1$-distances between the standard normal density and local U-statistics that were based on standardized observations. These tests were used to determine whether or not the data was normally distributed. Ref. [14] worked on the estimate of the mean of multivariate functions under the premise of possibly heavy-tailed distributions and presented the U-based median-of-means. U-processes are also necessary for a wide variety of statistical applications, such as the examination of qualitative aspects of functions in nonparametric statistics [15,16] as well as establishing limiting distributions of M-estimators (see, for example, [17,18,19]). In [20] the authors consider the problem of detecting distributional changes in a sequence of high dimensional data by using the weighted cumulative sums of U-statistics stemming from $L_p$ norms. In [21], the authors proposed tests based on U-statistics for testing the equality of marginal density functions. In the paper [22], the following problem is considered: Is it possible, given a sample of random variables that are independent, identically distributed, and have a finite variance, to build an estimator of the unknown mean that performs almost as well as if the data were normally distributed? The argument that was presented in the previous work is based on a new deviation inequality for the U-statistics of order that is permitted to rise with sample size. This inequality is the most important part of the argument. The first asymptotic results for the scenario in which the underlying random variables are assumed to be independent and distributed in an identical fashion were presented by [23,24] and [8] (amongst others), respectively. In contrast, the asymptotic results under weak dependency assumption were demonstrated in [25], in [26] or more recently in [27] and in more general settings in [28,29,30,31]. The interested reader may refer to [17,32] for an excellent collection of references on U-statistics and U-processes. We also refer to [19] for a profound understanding of the theory of U-processes.

In the present work, we consider the conditional U-statistics introduced by [33], that can be considered as generalizations of the Nadaraya-Watson ([34,35]) regression function estimates. To be more precise, let us consider the sequence of independent and identically distributed random vectors $\{({\mathbf{X}}_i,{\mathbf{Y}}_i),i\in {\mathbb{N}}^{*}\}$ with ${\mathbf{X}}_i\in {\mathbb{R}}^d$ and ${\mathbf{Y}}_i\in {\mathbb{R}}^{d^{\prime }}$, $d,d^{\prime }\ge 1$. Let $\phi :{\mathbb{R}}^{d^{\prime }k}\to \mathbb{R}$ denote a measurable function. Within the scope of this work, our primary focus is on the estimation of the conditional expectation or regression function, as follows:

$$\begin{array}{ccc}\hfill r^{\left(k\right)}(\phi ,\mathbf{t})& =& \mathbb{E}\left(\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t}\right),\mathrm{f}\mathrm{o}\mathrm{r}\mathbf{t}\in {\mathbb{R}}^{dk},\hfill \end{array}$$

whenever it exists, i.e., $\mathbb{E}\left(\left|\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)\right|\right)<\infty$. Now, we are going to present a kernel function $K:{\mathbb{R}}^d\to \mathbb{R}$ with support contained in ${[-B,B]}^d$, $B>0,$ fulfilling:

$$\underset{\mathbf{x}\in {\mathbb{R}}^d}{sup}\left|K\left(\mathbf{x}\right)\right|=:\kappa <\infty \mathrm{a}\mathrm{n}\mathrm{d}\int K\left(\mathbf{x}\right)d\mathbf{x}=1.$$

(1)

Ref. [33] established a new category of estimators for $r^{\left(k\right)}(\phi ,\mathbf{t})$, called conditional U-statistics, that is defined for each $\mathbf{t}\in {\mathbb{R}}^{dk}$ to be:

$$\begin{array}{c}\hfill {\widehat{\widehat{r}}}_n^{\left(k\right)}(\phi ,\mathbf{t};h_n)=\frac{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}\phi (Y_{i_1},\dots ,Y_{i_k})K\left(\frac{{\mathbf{t}}_1-{\mathbf{X}}_{i_1}}{h_n}\right)\cdots K\left(\frac{{\mathbf{t}}_k-{\mathbf{X}}_{i_k}}{h_n}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}K\left(\frac{{\mathbf{x}}_1-{\mathbf{X}}_{i_1}}{h_n}\right)\cdots K\left(\frac{{\mathbf{x}}_k-{\mathbf{X}}_{i_k}}{h_n}\right)}},\end{array}$$

(2)

where:

$$I(k,n)=\left\{\mathbf{i}=(i_1,\dots ,i_k):0\le i_j\le n\mathrm{a}\mathrm{n}\mathrm{d}{i}_j\ne i_r\mathrm{i}\mathrm{f}j\ne r\right\},$$

is the set of all k-tuples of different integers between 1 and n and ${\left\{h_n\right\}}_{n\ge 1}$ is a sequence of positive constants that, at a certain rate, converge to the value zero, $n{h}_n^{dk}\to \infty$. In the particular case $k=1$, the $r^{\left(k\right)}(\phi ,\mathbf{t})$ reduces to $r^{\left(1\right)}(\phi ,\mathbf{t})=\mathbb{E}\left(\phi \left(\mathbf{Y}\right)\right|\mathbf{X}=\mathbf{t})$ and the estimator developed by Stute is now known as the Nadaraya-Watson estimator $r^{\left(1\right)}(\phi ,\mathbf{t})$, given by:

$${\widehat{\widehat{r}}}_n^{\left(1\right)}(\phi ,\mathbf{t},h_n)=\sum _{i=1}^n\phi \left({\mathbf{Y}}_i\right)K\left(\frac{{\mathbf{X}}_i-\mathbf{t}}{h_n}\right)/\sum _{i=1}^{n}K\left(\frac{{\mathbf{X}}_i-\mathbf{t}}{h_n}\right).$$

The work of [36] focused on the estimation of the rate of the uniform convergence in $\mathbf{t}$ of ${\widehat{\widehat{r}}}_n^{\left(k\right)}(\phi ,\mathbf{t};h_n)$ to $r^{\left(k\right)}(\phi ,\mathbf{t})$. In [37], the limit distributions of ${\widehat{\widehat{r}}}_n^{\left(k\right)}(\phi ,\mathbf{t};h_n)$ are analyzed and compared to those produced by Stute. Under suitable mixing conditions, ref. [38] extend the results of [33] to weakly dependent data and uses their findings to evaluate the Bayes risk consistency of the corresponding discriminating rules.

As alternatives to the standard kernel-type estimators, [39] presented symmetrized nearest neighbor conditional U-statistics. This work has observed a major advancement because of the contributions of [40], where a far more strong version of consistency can be found; to be specific, uniform in $\mathbf{t}$ and in bandwidth consistency (i.e., $h_n$, $h_n\in [a_n,b_n]$ where $a_n<b_n\to 0$ at some specific rate) of ${\widehat{\widehat{r}}}_n^{\left(k\right)}(\phi ,\mathbf{t};h_n)$. Additionally, uniform consistency is achieved across $\phi \in \mathcal{F}$ for a suitably restricted class of functions $\mathcal{F}$, extended in [41,42,43,44] and [45]. The key component of their findings is the local conditional U-process studied in [11].

The case of functional data is the primary focus of this research. We present an excerpt from [46]: "Functional data analysis (FDA) is a branch of statistics concerned with analyzing infinite-dimensional variables such as curves, sets, and images. It has undergone phenomenal growth over the past 20 years, stimulated partly by significant data collection technology advances that have brought about the “Big Data” revolution. Often perceived as a somewhat arcane area of research at the turn of the century, FDA is now one of the most active and relevant fields of investigation in data science”. For an introduction to the subject of FDA, we refer to the books of [47,48], which contain different case studies in economics, archaeology, criminology, and neurophysiology, as well as fundamental analysis techniques. It is important to note that the extension of probability theory to random variables with values in normed vector spaces (such as Banach and Hilbert spaces), in conjunction with extensions of certain classical asymptotic limit theorems, predates the recent literature on functional data. This fact can be demonstrated by tracing back the history of the subject (see for instance, [49]). Ref. [50] investigated density and mode estimation for data with values in a normed vector space. At the same time, he brought attention to the issue of the curse of dimensionality, which affects functional data, and he suggested potential remedies to the problem. In the context of regression estimation, ref. [48] considered the nonparametric models. We may refer also to [51,52,53]. Recently, Modern theory has been applied to the treatment of functional data. For instance, ref. [54] provided the consistency rates of several functionals of the conditional distribution, such as the regression function, the conditional cumulative distribution, and the conditional density, amongst others, uniformly over a subset of the explanatory variable. Other examples include conditional density, which is a measure of the density of the conditional distribution, and conditional cumulative distribution, which is a measure of the conditional cumulative distribution. [55] also investigated the consistency rates for some functionals nonparametric models, such as the regression function, the conditional distribution, the conditional density, and the conditional hazard function, uniformly in bandwidth (UIB consistency) extended to the ergodic setting by [56]. In the paper [57], the topic of local linear estimation of the regression function in the case when the regressor is functional was investigated, and the results indicated robust convergence (with rates) consistently across bandwidth parameters. In the work of [58], the k-nearest neighbors (kNN) estimate of the nonparametric regression model for heavy mixing of functional time series data was explored. Under some mild conditions, a uniform and practically perfect convergence rate of the k-nearest neighbors estimator was established. In the work [59], the authors offer a variety of solutions for limiting laws for the conditional mode in the functional setting for ergodic data; for some current references, see the following: [45,60,61,62,63,64,65].

We will consider a general method for functional estimation by using the delta sequences. Delta sequences (also called “approximate identities” or “summability kernels”) arise in a wide variety of subfields within mathematics. Still, the applications that pertain to the theory of generalized functions are likely the most significant ones. The regularization of generalized functions is the major application for delta sequences. The proposed methods generalize several nonparametric estimation methods, including the kernel estimators given in (2) of [33]. To be more precise, the broad class of delta-sequence estimators includes the histogram estimators, Chentsov’s projection estimators [66], and nearest-neighbor estimators, among others. Certain types of these sequences were already studied by [67], who called them “$\delta$-function sequences”. They established, among other things, the asymptotic unbiasedness and the asymptotic variance of estimators based on them but did not consider convergence rates. Ref. [68] obtained the rate of strong consistency and the rate of asymptotic bias for estimators associated with delta sequences arising from the Fejér kernel of the Fourier series. The delta sequence method of density estimation of [69] is extended to certain non-i.i.d. cases in [70], where it is assumed that the observations are taken from a stationary Markov process. Ref. [71] considered the delta-sequence estimator for the marginal distribution of a strictly stationary stochastic process satisfying some mixing conditions. In [72], the author investigated the local and global convergence rates of delta-sequence type estimators of the density function, its derivative, and its mode. Ref. [73] proved the uniform strong consistency of delta-sequence estimators. Ref. [74] partially generalized the usual nonparametric estimators of a regression function by using an estimator based on quasi-positive delta sequences. Ref. [75] considered a general nonparametric statistical curve estimation setting called the class of “fractional delta sequence estimators”. Ref. [76] used the delta method to investigate the correlation model. [77] looked at the problem of estimating the density function of functional data with values in an infinite-dimensional separable Banach space using the method of delta sequences; for further information, we can also look into [78]. The copula estimation using the delta sequences methods is considered in [79]. The problem of the nonparametric minimax estimation of a multivariate density at a given point by the delta sequences was investigated in [80]. Ref. [81] used the delta sequence to propose an essential application to the classification problem of the value of the discrete random variable.

The goal of the current study is to present and investigate the delta sequences estimators for the conditional U-statistics for functional data, more specifically for random elements taking values in an infinite-dimensional separable Banach space, such as the space of continuous functions on the interval [0, 1] endowed with the supremum norm. This will allow the delta sequences estimators for the conditional U-statistics for functional data to be utilized for functional data analysis. Examples of functional data that can appear in these spaces include stochastic processes with continuous sample paths on a finite interval associated with the supremum norm and stochastic processes whose sample paths are square-integrable on the real line. Both of these types of stochastic processes can occur on the real line. The dimensionality problem must be addressed in a nonparametric functional data analysis in two ways: first, by working with data that have an infinite number of dimensions, and second, by making universal assumptions about the infinite number of dimensions for the probability distribution of variables in nonparametric modeling. This structure’s twofold infinity of dimensions is the basis for all subsequent developments in the discipline. Our previous work, delivered in the multivariate setting and cited as [82], is extended here in the present study. Although the concept behind our estimation approach is similar to that presented in [82] (containing the Stute estimator), we make allowances for the infinite dimensionality of the covariate. have determined the asymptotic characteristics of the multivariate delta sequence estimators, ref. [82]. Their findings do not directly apply to the current situation since we are working with a covariate with an unlimited number of dimensions. As a result, we must utilize other reasoning in our proofs to deal with the broader framework. These findings are beneficial in their own right, but they are also necessary for the inquiry being conducted in this work. To “simply” combine ideas from other publications would not be sufficient to solve the issue, as will be demonstrated in the following paragraphs. To be able to deal with delta sequence U-statistic estimators for functional data, you will need to resort to intricate mathematical derivations. Compared to the previous studies written on delta sequence estimators, the current paper considers the setting of an unbounded function $\phi$, which adds a significant amount of complexity to the proof. The general assumptions that are required for the derivations of the asymptotic results for the conditional U-statistics delta sequence estimators are presented in this study.

The format of this article is structured as follows. Section 2 is devoted to introducing the delta sequences and the definitions we need in our work, where we introduce the new family of estimators. Section 3 gives the paper’s main results concerning the uniform convergences. In Section 4, we present a significant application for the censored data context of its interest. In Section 5, we provide some applications, including the Kendall rank correlation coefficient in Section 5.1, the discrimination in Section 5.2, the the metric learning in Section 5.3 and the time series prediction from a continuous set of past values in Section 5.4. Some final observations and possible developments in the future are moved to Section 6. To maintain a smooth flow throughout the presentation, all proofs have been compiled in Section 7. A selection of significant technical findings is presented in Appendix A.

2. Preliminaries and Estimation Procedure

Let $(\Omega ,\mathcal{F},\mathbb{P})$ denote a probability space, $(\mathcal{X},d(·,·\left)\right)$ denote an infinite-dimensional separable Banach space equipped with a norm $\parallel .\parallel$ such that $d(u,v)=\parallel u-v\parallel$ and $\mathcal{B}$ be the $\sigma$-algebra of Borel subsets of $\mathcal{X}$. Let us consider a sequence $\{X_i,Y_i:i\ge 1\}$ of independent identically distributed random copies of the random element $(X,Y)$, where X is a random element defined on $(\Omega ,\mathcal{F},\mathbb{P})$ taking values in $(\mathcal{X},\mathcal{B})$ and Y takes values in some abstract space $(\mathcal{Y},{\mathcal{B}}^{\prime })$. Ref. [83] introduced the functional conditional U-statistics when $\mathbf{x}\in {\mathcal{X}}^m$ some semi-metric space as a generalization of Stute’s estimator by:

$$\begin{array}{c}\hfill {\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{x};h_K)=\frac{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}\phi (Y_{i_1},\dots ,Y_{i_m})K\left(\frac{d(x_1,X_{i_1})}{h_K}\right)\cdots K\left(\frac{d(x_m,X_{i_m})}{h_K}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d(x_1,X_{i_1})}{h_K}\right)\cdots K\left(\frac{d(x_m,X_{i_m})}{h_K}\right)}}.\end{array}$$

(3)

As we mentioned, the delta-sequences procedures can be considered a more general class, including kernel estimation techniques. Therefore, we can naturally obtain a more general class of functional conditional U-statistics by replacing the kernel $K(·)$ in Equation (3) with positive delta sequences ${\delta }_m(·,·)$ (see Definition 1), which allows us to introduce the following conditional U-statistic for each $\mathbf{x}=(x_1,\dots ,x_k)\in {\mathcal{X}}^k$ and $\phi :{\mathcal{Y}}^k\to \mathbb{R}$ a measurable function, by

$$\begin{array}{c}\hfill {\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)=\left\{\begin{array}{c}{\displaystyle \frac{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}\phi (Y_{i_1},\dots ,Y_{i_k}){\delta }_{m_n}\left(x_1,X_{i_1}\right)\cdots {\delta }_{m_n}\left(x_k,X_{i_k}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}{\delta }_{m_n}\left(x_1,X_{i_1}\right)\cdots {\delta }_{m_n}\left(x_k,X_{i_k}\right)}},}\hfill \\ \\ {\displaystyle \mathrm{i}\mathrm{f}\sum _{(i_1,\dots ,i_k)\in I(k,n)}{\delta }_{m_n}\left(x_1,X_{i_1}\right)\cdots {\delta }_{m_n}\left(x_k,X_{i_k}\right)\ne 0,}\hfill \\ \\ {\displaystyle \frac{n!}{(n-k)!k!}\sum _{(i_1,\dots ,i_k)\in I(k,n)}\phi (Y_{i_1},\dots ,Y_{i_k})}\hfill \\ \\ {\displaystyle \mathrm{i}\mathrm{f}\sum _{(i_1,\dots ,i_k)\in I(k,n)}{\delta }_{m_n}\left(x_1,X_{i_1}\right)\cdots {\delta }_{m_n}\left(x_k,X_{i_k}\right)=0,}\hfill \end{array}\right.\end{array}$$

(4)

which we consider estimating the regression function

$$r^{\left(k\right)}(\phi ,\mathbf{x})=\mathbb{E}\left(\phi (Y_1,\dots ,Y_k)\mid (X_1,\dots ,X_k)=\mathbf{x}\right),\mathrm{f}\mathrm{o}\mathrm{r}\mathbf{x}\in {\mathcal{X}}^k,$$

(5)

whenever it exists, i.e, ${\displaystyle \mathbb{E}\left(\left|\phi (Y_1,\dots ,Y_k)\right|\right)<\infty }$.

Remark 1. 

It is worth noting that X may admit a probability density function $f(·)$ in relation to a σ-finite measure μ on $(\mathcal{X},\mathcal{B})$ in such a way that (for instance, refer [77,78,84]):

$$\mathbb{P}(X\in A)={\int }_{A}f\left(x\right)\mu \left(dx\right),for everyA\in \mathcal{B}such that0<\mu \left(A\right)<\infty .$$

(6)

The concept of this remark is elaborated upon in [77] and its references. We denote by $(\Omega ,\mathcal{F},\mathbb{P})$ a probability space and a nondecreasing family of sub- $\sigma$-algebras of $\mathcal{F}$ is denoted by ${\left\{{\mathcal{F}}_t\right\}}_{t\ge 0}$. Let ${\left\{W\left(t\right)\right\}}_{t\ge 0}$ denote a standard Wiener process defined on $(\Omega ,\mathcal{F},\mathbb{P})$ in such a way that $W_t$ is ${\mathcal{F}}_t$-measurable. We highlight that the probability measure ${\mu }_W$ on the space ${\mathcal{C}}^0(0,T)$ is connected with a Borel $\sigma$-algebra generated by the supremum norm topology is induced by the standard Wiener process. Let ${\left\{X\left(t\right)\right\}}_{0\le t\le T}$ be a diffusion process defined the stochastic differential equation:

$$dX\left(t\right)=a(t,X(t\left)\right)dt+b(t,X(t\left)\right)dW\left(t\right),$$

where $X\left(0\right)=x_0$ for $0\le t\le T$. By imposing some assumptions on the functions $a(·,·)$ and $b(·,·)$, we can establish that the probability measure ${\mu }_X$ on the space ${\mathcal{C}}^0(0,T)$ induced by the process X is absolutely continuous with respect to the probability measure ${\mu }_W$. In addition, applying Girsanov’s Theorem permits the computation of the Radon-Nikodym derivative of ${\mu }_X$ with respect to ${\mu }_W$. The probability density of X on the space ${\mathcal{C}}^0(0,T)$ is the ${\mu }_W$ derivative, for instance, see [85]. From this point of view, the main motivation leading to the analysis of functional data is the inference of stochastic processes; the reader is referred to [85,86]. For the purposes of drawing conclusions, we make the assumption that the entirety of the process can be observed. However, if the process can only be observed at discrete times, either on a tiny grid or when the data are sparse, then other approaches, such as parametric inference for discrete data, need to be devised. For instance, for the diffusion processes, these new methods are necessary (cf. [85,87,88]). Observe that if $A=B(x,\kappa )$ for $(x,\kappa )\in \mathcal{X}\times {\mathbb{R}}_{+}^{*}$, then (6) allows for the small ball probability to be considered.

Assume also that $S_{\mathcal{X}}$ is a pseudo-compact subset of $\mathcal{X}$ satisfying the following property: for any $ϵ>0$, there exists $t_{\ell }\in \mathcal{X},1\le \ell \le d_n$ such that

$$S_{\mathcal{X}}\subset {\mathcal{S}}_n:=\bigcup _{\ell =1}^{d_n}B(t_{\ell },ϵ),$$

(7)

and there exists $\kappa >0$ such that $d_n{ϵ}^{\kappa }$ is a constant $C>0$. Here, the open ball with center $t_{\ell }$ and radius $ϵ$ is denoted by $B\left(t_{\ell },ϵ\right)$.

It is worth mentioning that the hypothesis (7) is essential for assuming a geometrical link between the number $d_n$ of balls and their radius $ϵ$. In addition, this condition is fulfilled in usual nonparametric problems when $\mathcal{X}={\mathbb{R}}^p$ is endowed with the Euclidean metric on ${\mathbb{R}}^p$ (because $\kappa =p$ suffices). However, this topological characteristic does not hold for any abstract semi-metric space, as [89] explains. Before we can use the delta-sequences approach to estimate the value of the regression operator $r^{\left(k\right)}(·)$ in the model (5), we must first have the following definition.

Definition 1. 

A sequence of non-negative functions $\left\{{\delta }_{m_n}(x,y):m_n\ge 1\right\}=\left\{{\delta }_m(x,y),m\ge 1\right\}$ defined on ${\mathcal{X}}^k\times {\mathcal{X}}^k$ is called a delta-sequence with respect to the measure μ if the following properties are satisfied:

(C.1) 

For each γ in such a way that $0<\gamma \le \infty$:

$$\underset{m\to +\infty }{lim}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|{\int }_{\mathbb{B}(\mathbf{x},\gamma )}{\delta }_m(\mathbf{x},\mathbf{y})\mu \left(d\mathbf{y}\right)-1\right|=0,$$

(8)

where ${\displaystyle \mathbb{B}(\mathbf{x},\gamma ):=\prod _{j=1}^{k}B(x_j,\gamma )}$, for all $\mathbf{x}=(x_1,\dots ,x_k)$.

(C.2) 

There exists a positive constant $C_1$, in such a way that

$$\underset{(\mathbf{x},\mathbf{y})\in S_{\mathcal{X}}^k\times {\mathcal{X}}^k}{sup}{\delta }_m(\mathbf{x},\mathbf{y})\le C_1{s}_m<\infty ,$$

(9)

where $0<s_m\to \infty$ as $m\to \infty$ and ${\displaystyle \underset{m\to \infty }{lim}\frac{m}{s_{m}log\left(m\right)}=\infty }$.

(C.3) 

There exist $C_2>0,{\beta }_1>0$ and ${\beta }_2>0$, in such a way that

$$\left|{\delta }_m({\mathbf{x}}_1,\mathbf{y})-{\delta }_m({\mathbf{x}}_2,\mathbf{y})\right|\le C{s}_m^{{\beta }_2}d{({\mathbf{x}}_1,{\mathbf{x}}_2)}^{{\beta }_1}forall{\mathbf{x}}_1,{\mathbf{x}}_2,\mathbf{y}\in {\mathcal{X}}^k,$$

(10)

where

$$d(\mathbf{x},\mathbf{y}):=\frac{1}{k}d\left(x_1,y_1\right)+\cdots +\frac{1}{k}d\left(x_k,y_k\right),$$

for all $\mathbf{x}=(x_1,\dots ,x_k)$ and $\mathbf{y}=(y_1,\dots ,y_k)\in {\mathcal{X}}^k.$

(C.4) 

For any $\gamma >0$:

$$\underset{m\to \infty }{lim}\underset{\begin{array}{c}\mathbf{x}\in S_{\mathcal{X}}^k\\ \mathbf{y}\in \overline{\mathbb{B}}(x,\gamma )\end{array}}{sup}{\delta }_m(\mathbf{x},\mathbf{y})d(\mathbf{x},\mathbf{y})=0,$$

(11)

where the complement set of the open ball $\mathbb{B}(\mathbf{x},\gamma )$ is denoted by $\overline{\mathbb{B}}(\mathbf{x},\gamma )$.

Notice that the conditions (C.1)–(C.4) of Definition 1 are modelled after a similar set of conditions for kernel-type estimators. The condition (C.2) corresponds to the bound of ${\delta }_m$ over $S_{\mathcal{X}}^k\times {\mathcal{X}}^k$ whereas the condition (C.3) is pertains to the uniform Lipschitz property of ${\delta }_m(\mathbf{x},\mathbf{y})$. Contrarily, the condition (C.4) is and it is not an assumption on the bound of $d(\mathbf{x},\mathbf{y})$ over $S_{\mathcal{X}}^k\times S_{\mathcal{X}}^k$ but an assumption on the limiting behaviour of ${\delta }_m(\mathbf{x},\mathbf{y})d(\mathbf{x},\mathbf{y})$ as $m\to +\infty$.

Proposition 1. 

Let $\left\{{\delta }_{m,1}(x_1,y_1)\right\},\dots ,\left\{{\delta }_{m,k}(x_k,y_k)\right\}$ each be non-negative delta-sequence with respect to the measure μ, then

$${\delta }_m(\mathbf{x},\mathbf{y}):=\prod _{j=1}^k{\delta }_{m,j}(x_j,y_j),$$

(12)

is also a non-negative delta sequence.

This proposition is similar to Proposition 2.2 [69] when $\mathcal{X}=\mathbb{R}{}^d$, which means that the product of non-negative delta-sequences is also a positive delta sequence. The proposition provides a flexible way to construct delta sequences in high dimensions in a similar way to kernel type estimation. Unless otherwise specified, we will set

$${\displaystyle {\delta }_m(\mathbf{x},\mathbf{y}):=\prod _{j=1}^k{\delta }_{m_n}(x_j,y_j)}$$

for all $\mathbf{x}$ and $\mathbf{y}\in {\mathcal{X}}^k$. This notation will unburden our results in the forthcoming theorems.

2.1. Examples of Delta Sequence

In this section, following the notation of [78], we provide guidelines for constructing and recovering some well-known estimators in literature.

Example 1. 

Kernel estimator

Let $\mathcal{X}={\mathcal{C}}^0(0,1)$ denote the space of the real-valued continuous functions that vanishes at 0. Suppose that $\mathcal{X}$ is equipped with the uniform topology that is induced by the supremum norm, i.e., if $x\in {\mathcal{C}}^0(0,1)$ then x is continuous on $(0,1)$ with $x\left(0\right)=0$ and that

$$\parallel x\parallel =\underset{t\in (0,1)}{sup}\left|x\left(t\right)\right|.$$

The Wiener measure on the space $\mathcal{X}$ induced by the standard Wiener process is denoted by μ. Let us define

$${\delta }_m(x,y)=\frac{1}{\mu \left(B\right(x,1/m\left)\right)}{\mathbb{𝟙}}_{B(x,1/m)}\left(y\right),$$

where as usual ${{\displaystyle \mathbb{𝟙}}}_A$ denotes the indicator function of the set A. Set for all $\mathbf{x}=(x_1,\dots ,x_k)$ and $\mathbf{y}=\mathbf{x}=(y_1,\dots ,y_k)\in {\mathcal{X}}^k$

$${\delta }_m(\mathbf{x},\mathbf{y})=\prod _{j=1}^k{\delta }_m(x_j,y_j),$$

then, by Proposition 1, ${\delta }_m(\mathbf{x},\mathbf{y})$ is a non-negative delta sequence, and the conditional U-statistic is defined in this case by

$$\begin{array}{cc}\hfill {\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)& =\frac{{\displaystyle \sum _{i\in I(k,n)}\phi (Y_{i_1},\dots ,Y_{i_k})\prod _{j=1}^k{\mathbb{𝟙}}_{B(x_j,1/m)}\left(X_{i_j}\right)}}{{\displaystyle \sum _{i\in I(k,n)}\prod _{j=1}^k{\mathbb{𝟙}}_{B(x_j,1/m)}\left(X_{i_j}\right)}}\hfill \\ \hfill & =\frac{{\displaystyle \sum _{i\in I(k,n)}\phi (Y_{i_1},\dots ,Y_{i_k})\prod _{j=1}^k{\mathbb{𝟙}}_{B(x_j,1)}\left(\frac{d(x_j,X_{i_j})}{1/m}\right)}}{{\displaystyle \sum _{i\in I(k,n)}\prod _{j=1}^k{\mathbb{𝟙}}_{B(x_j,1)}\left(\frac{d(x_j,X_{i_j})}{1/m}\right)}},\hfill \end{array}$$

which can be considered as the naive kernel estimator of $r^{\left(k\right)}(·)$. We can observe clearly that ${\delta }_m(·,·)$ in this example satisfies the condition (C.1). In fact

$$\begin{array}{cc}\hfill & \underset{m\to +\infty }{lim}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|{\int }_{\mathbb{B}(\mathbf{x},\gamma )}\prod _{j=1}^k\frac{1}{\mu \left(B(x_j,1/m)\right)}{\mathbb{𝟙}}_{B(x_j,1)}\left(\frac{d(x_j,y_j)}{1/m}\right)\mu \left(d{y}_1\right)\cdots \mu \left(d{y}_k\right)-1\right|\hfill \\ \hfill & =\underset{m\to +\infty }{lim}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\prod _{j=1}^k\frac{1}{\mu \left(B(x_j,1/m)\right)}\mu \left(B(x_j,\gamma )\cap B(x_j,1/m)\right)-1\right|\hfill \\ \hfill & =\underset{m\to +\infty }{lim}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\prod _{j=1}^k\frac{1}{\mu \left(B(x_j,1/m)\right)}\mu \left(B\left(x_j,min(\gamma ,1/m)\right)\right)-1\right|,\hfill \end{array}$$

this quantity tends to zero when m is sufficiently large.

For a bandwidth $h_n^x$, that is a sequence of positive numbers, define

$${\delta }_m(x,y)=\frac{1}{h_n^x}{K}_n\left(d(x,y)\right),$$

where $K_n(·)$ is a sequence of functions fullfilling (C.1)–(C.4).

Example 2. 

Histogram estimator

Let ${\mathcal{P}}_n=\left\{A_{n,j},j\in J_n\right\}$ be a partition of the set F (cf. [90]), such that

$$\left|J_n\right|=m_n,\underset{j\in J_n}{max}\mu \left(A_{n,j}\right)\to 0andn\underset{j\in J_n}{min}\mu \left(A_{n,j}\right)\to \infty asn\to \infty .$$

Denote

$${\delta }_m(x,y)=\sum _{j\in J_n}\frac{1}{\mu \left(A_{n,j}\right)}{\mathbb{𝟙}}_{A_{n,j}}\left(x\right){\mathbb{𝟙}}_{A_{n,j}}\left(y\right).$$

We can now construct the histogram and regressogram estimators in the conditional U-statistics framework by taking ${\displaystyle {\delta }_m(\mathbf{x},\mathbf{y})=\prod _{i=1}^k{\delta }_m(x_i,y_i)}$.

Example 3. 

Orthogonal series estimator

Let ${\left\{e_p\right\}}_{p\ge 1}$ be a complete orthonormal system of the space $\mathcal{X}$, comprising eigenfunctions of a compact operator in the square integral functions space $\left(L_2\left(\mathcal{X}\right)\right.$, say). Define

$${\delta }_m(x,y)=\sum _{p=1}^m{e}_p\left(x\right)e_p\left(y\right)forx,y\in F.$$

As stated in [91], ${\delta }_m(·,·)$ in this case are delta sequences. Now using Proposition 1, we can observe that

$${\displaystyle {\delta }_m(\mathbf{x},\mathbf{y})=\prod _{i=1}^k{\delta }_m(x_i,y_i)}$$

are also a positive delta sequences.

For more examples of delta sequences, we refer to [69,92].

2.2. Conditions and Comments

In order to study the consistency of the proposed estimator, let us first state the following conditions:

(C.5) 

We assume that $d_n=n^{\zeta }$ for $\zeta >0$ and

$${ϵ}^{{\beta }_1}{s}_m^{{\beta }_2}<\sqrt{\frac{s_{m}log\left(m\right)}{m}}.$$

(C.6) 

Suppose that $m\to \infty$ and that

$$\exists 0<\tau <1\mathrm{in}\mathrm{such}\mathrm{a}\mathrm{way}\mathrm{that}{n}^{\tau }\le m\le n,\mathrm{for}\mathrm{large}n.$$

(C.7) 

We assume the following usual boundedness condition:

$$\underset{\mathbf{y}\in {\mathcal{Y}}^k}{sup}\left|\phi \left(\mathbf{y}\right)\right|=M<\infty .$$

(C.7’) 

The function $\phi$ is unbounded and fullfils for some $q>2:$

$${\mu }_q:=\underset{\mathbf{t}\in S_{\mathcal{X}}^k}{sup}\mathbb{E}\left({\phi }^q\left(\mathbf{Y}\right)|\mathbf{X}=\mathbf{t}\right)<\infty .$$

(C.8) 

For every $\gamma \ge 0$:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|{\int }_{\overline{\mathbb{B}}(\mathbf{x},\gamma )}{\delta }_m(\mathbf{x},\mathbf{y})\mu \left(d\mathbf{y}\right)-1\right|=O\left(D_m\right),$$

where $D_m=\left\{d(\mathbf{x},\mathbf{y}),\mathbf{x}\in S_{\mathcal{X}}^k\mathrm{a}\mathrm{n}\mathrm{d}\mathbf{y}\in {\mathcal{X}}^k,\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h} \mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}{\delta }_m(\mathbf{x},\mathbf{y})>0\right\}=o\left(1\right)$ as $m\to$$+\infty$.

(C.9) 

The regression operator $r^{\left(k\right)}(\phi ,·)$ is Lipschitzian in the following sense: $\exists C_3>0$ in such a way that, for any ${\mathbf{x}}_1\in S_{\mathcal{X}}^k$ and ${\mathbf{x}}_2\in {\mathcal{X}}^k$, we have

$$|r^{\left(k\right)}(\phi ,{\mathbf{x}}_1)-r^{\left(k\right)}(\phi ,{\mathbf{x}}_2)|\le C_{3}d({\mathbf{x}}_1,{\mathbf{x}}_2).$$

2.3. Comments on the Assumptions

Similar to conditions (C.1)–(C.4), assumption (C.5) is also modelled after some kernel-type conditions, and it allows us to select ${\beta }_1$ and ${\beta }_2$ in condition (C.3). Due to the infinite nature of the problem, additional constraints are required to achieve uniform consistency across the pseudo-compact set. Ref. [89], discussed the assumption (7). This condition holds trivially for any finite-dimensional Euclidean space and remains valid for projection-based metric spaces with infinite dimensions. Condition (C.7) concerning the boundedness of the function $\phi (·)$ is essential to establish exponential bounds, this, coupled with the technical condition (C.6), allows us to obtain the almost complete convergence later in the proofs. Note that we can replace condition (C.7) with a more general one, that is, condition (C.7’), to obtain the results when the function $\phi (·)$ is unbounded. Finally, to establish precise rates of almost complete convergence in the functional context, additional conditions related to the topological nature of the problem are required. Mainly the assumption (C.8) and (C.9), where the latter condition concerning the Lipschitz property of the operator $r^{\left(k\right)}(·)$ is standard when studying with uniform consistency.

Remark 2. 

Note that the condition (C.7) can be replaced by more broad hypotheses at specific times of  $\mathbf{Y}$, as shown in [93]. That is

(C.7’) 

We denote by $\left\{\mathcal{M}\right(x):x\ge 0\}$ a nonnegative continuous function, increasing on $[0,\infty )$, and such that, for some $s>2$, ultimately as $x\uparrow \infty$,

$$\left(i\right)x^{-s}\mathcal{M}\left(x\right)\downarrow ;\left(ii\right)x^{-1}\mathcal{M}\left(x\right)\uparrow .$$

(13)

For each $t\ge \mathcal{M}\left(0\right)$, we define ${\mathcal{M}}^{inv}\left(t\right)\ge 0$ by $\mathcal{M}\left({\mathcal{M}}^{inv}\left(t\right)\right)=t$. We assume further that:

$$\mathbb{E}\left(\mathcal{M}\left(\left|\phi \left(\mathbf{Y}\right)\right|\right)\right)<\infty .$$

The following choices of $\mathcal{M}(·)$ are of particular interest:

(i)

$\mathcal{M}\left(x\right)=x^p$ for some $p>2$;

(ii)

$\mathcal{M}\left(x\right)=exp\left(sx\right)$ for some $s>0$.

The boundedness assumption on $\phi (·)$ can be substituted by a finite moment assumption (C.7’), but doing so will add a significant amount of additional complexity to the proofs; for further information, check the most recent reference [42,45,62,94,95] for more details.

3. Some Asymptotic Results

In this subsection, we will discuss the uniform consistency of the functional conditional U-statistic, which is defined by (4). First, let us provide basic notation

$$\begin{array}{ccc}\hfill \mathbf{X}& :=& (X_1,\dots ,X_k)\in {\mathcal{X}}^k,\mathbf{Y}:=(Y_1,\dots ,Y_k)\in {\mathcal{Y}}^k,\hfill \\ \hfill {\mathbf{X}}_i& :=& (X_{i_1},\dots ,X_{i_k}),{\mathbf{Y}}_i:=(Y_{i_1},\dots ,Y_{i_k}),\hfill \\ \hfill G_{\phi ,\mathbf{x}}(\mathbf{X},\mathbf{Y})& :=& \phi \left(\mathbf{Y}\right){\delta }_m(\mathbf{x},\mathbf{X})\mathrm{f}\mathrm{o}\mathrm{r}\mathbf{x}\in S_{\mathcal{X}}^k,\hfill \\ & & \\ \hfill u_n(\phi ,\mathbf{x},m_n)& =& u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}\right):=\frac{(n-k)!}{n!}\sum _{i\in I(k,n)}{G}_{\phi ,\mathbf{x}}({\mathbf{X}}_i,{\mathbf{Y}}_i).\hfill \end{array}$$

It is clear that, for all $\mathbf{x}\in {\mathcal{X}}^k:$

$${\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)=\frac{u_n(\phi ,\mathbf{x},m_n)}{u_n(1,\mathbf{x},m_n)},$$

and $u_n(\phi ,\mathbf{x},m_n)$ is a classical U-statistic with the U-kernel $G_{\phi ,\mathbf{x},m_n}(\mathbf{x},\mathbf{y}).$ Therefore, to establish the uniform consistency of ${\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)$ to $r^{\left(k\right)}(\phi ,\mathbf{x})$ we need to study the uniform consistency of $u_n(\phi ,\mathbf{x},m_n)$ to $\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)$. In this case, we will be considering a suitable centering parameter different from the expectation $\mathbb{E}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)$; hence, we define:

$$\widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)=\frac{\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)}{\mathbb{E}\left(u_n(1,\mathbf{x},m_n)\right)}.$$

(14)

The notation and facts that are presented below should be included in the continuation of this discussion. For a kernel L of $k\ge 1$ variables, we define

$$U_n^{\left(k\right)}\left(L\right)=\frac{(n-k)!}{n!}\sum _{i\in I(k,n)}L\left(X_{i_1},\dots ,X_{i_k}\right)$$

Suppose that L is a function of $\ell \ge 1$ variables, symmetric in its entries. Then, the Hoeffding projections (see [8,19]) with respect to $\mathbb{P}$, for $1\le k\le \ell$, are defined as

$${\pi }_{k,\ell }L\left(x_1,\dots ,x_k\right)=\left({\Delta }_{x_1}-\mathbb{P}\right)\times \cdots \times \left({\Delta }_{x_k}-\mathbb{P}\right)\times {\mathbb{P}}^{\ell -k}\left(L\right),$$

and

$${\pi }_{0,\ell }L=\mathbb{E}L\left(X_1,\dots ,X_{\ell }\right),$$

for some measures $Q_i$ on S, we denote

$$Q_1\cdots Q_{k}L={\int }_{S^k}L(x_1,\dots ,x_k)d{Q}_1\left(x_1\right)\cdots d{Q}_k\left(x_k\right),$$

and ${\Delta }_x$ denote Dirac measure at point $x\in \mathcal{X}$. Then, the Hoeffding decomposition give the following

$$U_n^{(\ell )}\left(L\right)-\mathbb{E}L=\sum _{k=1}^{\ell }\left(\begin{array}{c}\ell \\ k\end{array}\right)U_n^{\left(k\right)}\left({\pi }_{k,\ell }L\right),$$

which is easy to check. For $L\in L_2\left({\mathbb{P}}^{\ell }\right)$ this denotes an orthogonal decomposition and $\mathbb{E}({\pi }_{k}L\mid X_2,\dots ,X_k)=0$ for $k\ge 1$; which is, the kernels ${\pi }_{k,\ell }L$ are canonical for $\mathbb{P}$. Moreover, ${\pi }_{k,\ell },k\ge 1$, are nested projections, that is, ${\pi }_{k,\ell }\circ {\pi }_{k^{\prime },\ell }={\pi }_{k,\ell }$ if $k\le k^{\prime }$, and

$$\mathbb{E}{\left({\pi }_{k,\ell }L\right)}^2\le \mathbb{E}{(L-\mathbb{E}L)}^2\le \mathbb{E}{L}^2.$$

For example,

$${\pi }_{1,\ell }h\left(x\right)=\mathbb{E}(h(X_1,\dots ,X_{\ell })\mid X_1=x)-\mathbb{E}h(X_1,\dots ,X_{\ell }).$$

Remark 3. 

The function $G_{\phi ,\mathbf{x},m_n}$ is not necessarily symmetric; when we need to symmetrize them, we have:

$${\overline{G}}_{\phi ,\mathbf{x}}(\mathbf{x},\mathbf{y}):=\frac{1}{k!}\sum _{\sigma \in I_k^k}{G}_{\phi ,\mathbf{x},m_n}({\mathbf{x}}_{\sigma },{\mathbf{y}}_{\sigma })=\frac{1}{k!}\sum _{\sigma \in I_k^k}\phi \left({\mathbf{y}}_{\sigma }\right){\delta }_{m_n}({\mathbf{x}}_{\sigma },{\mathbf{y}}_{\sigma }),$$

where ${\mathbf{x}}_{\sigma }=(x_{{\sigma }_1},\dots ,x_{{\sigma }_k})$ and ${\mathbf{y}}_{\sigma }=(y_{{\sigma }_1},\dots ,y_{{\sigma }_k})$. After symmetrization, the expectation

$$\mathbb{E}\left({\overline{G}}_{\phi ,\mathbf{x},m_n}(\mathbf{x},\mathbf{y})\right)=\mathbb{E}\left(G_{\phi ,\mathbf{x},m_n}(\mathbf{x},\mathbf{y})\right),$$

and the U-statistic

$$u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x},m_n}\right)=u_n^{\left(k\right)}\left({\overline{G}}_{\phi ,\mathbf{x},m_n}\right):=u_n(\phi ,\mathbf{x},m_n)$$

do not change.

3.1. Uniform Consistency of Functional Conditional U-Statistics

Let $\left(z_n\right)$ for $n\in \mathbb{N}$, be a sequence of real r.v.’s. We say that $\left(z_n\right)$ converges almost-completely (a.co.) toward zero if, and only if, for all

$$ϵ>0,\sum _{n=1}^{\infty }\mathbb{P}\left(\left|z_n\right|>ϵ\right)<\infty .$$

Moreover, we say that the rate of the almost-complete convergence of $\left(z_n\right)$ toward zero is of order $u_n$ (with $u_n\to 0$ ) and we write $z_n=O_{a.co.}\left(u_n\right)$ if, and only if, there exists $ϵ>0$ such that

$$\sum _{n=1}^{\infty }\mathbb{P}\left(\left|z_n\right|>ϵu_n\right)<\infty .$$

This kind of convergence implies both the almost-sure convergence and the convergence in probability. The following result concerns the uniform deviation of the estimate $u_n(\phi ,\mathbf{x},m_n)$ with respect to $\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)$ when the function $\phi$ is bounded.

Theorem 1. 

Under the conditions (C.1)–(C.4), and if conditions (C.5), (C.7) are satisfied, then we have:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

We present a more general result concerning the case when the function $\phi$ is unbounded in the sense of the condition (C.7’). That being said, the preceding theorem constitutes an important step in the truncation method used in the proof of the following theorem.

Theorem 2. 

Under the conditions (C.1)–(C.4), and if conditions (C.5), (C.6) and (C.7’) are satisfied, then we have:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

The following result handles the uniform deviation of the estimator ${\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)$ with respect to $\widehat{\mathbb{E}}\left[{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right]$ in the case of the function $\phi$ is bounded or unbounded.

Theorem 3. 

Under the conditions (C.1)–(C.4), and if conditions (C.5), (C.6) and condition (C.7) (or(C.7’)) are satisfied, then we have:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)-\widehat{\mathbb{E}}\left[{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right]\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right),$$

where ${\left(s_m\right)}_{m\in N^{*}}$ is a sequence of positive real numbers, in such a way that $m{\left(s_{m}log\left(m\right)\right)}^{-1}$$\to \infty$ as $n\to \infty$.

Theorem 4. 

Under the conditions (C.1)–(C.4) and (C.9), we have:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\widehat{\mathbb{E}}\left[{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right]-r^{\left(k\right)}(\phi ,\mathbf{x})\right|\to 0.$$

The following corollary is more or less straightforward, given Theorems 3 and 4.

Corollary 1. 

Under the conditions of Theorems 3 and 4 it follows that, as m tends to infinity:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)-r^{\left(k\right)}(\phi ,\mathbf{x})\right|\to 0,a.co.,$$

where ${\left(s_m\right)}_{m\in {\mathrm{N}}^{*}}$ is a sequence of positive real numbers, in such a way that $m{\left(s_{m}log\left(m\right)\right)}^{-1}$$\to \infty$ as $n\to \infty$.

3.2. Uniform Strong Consistency with Rates

This section is devoted to the uniform version with the rate of Theorem 1’s result. More specifically, our objective is to obtain the uniform almost-complete convergence of ${\widehat{r}}_n^{\left(k\right)}(·)$ on some subset $S_{\mathcal{X}}^k$ of ${\mathcal{X}}^k$ satisfying condition (7). In the following theorem, we establish the bias order.

Theorem 5. 

Under the conditions (C.1)–(C.4), and if conditions (C.8) and (C.9) are satisfied, then we have:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)-r^{\left(k\right)}\left(\phi ,\mathbf{x}\right)\right|=O\left(D_m\right).$$

(15)

The almost-complete convergence is then given by the corollary that follows, which uses a rate of ${\widehat{r}}_n^{\left(k\right)}(·)$.

Corollary 2. 

Under the conditions of Theorems 3 and 5 it follows that:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)-r^{\left(k\right)}(\phi ,\mathbf{x})\right|=O\left(D_m\right)+O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

(16)

4. Conditional $\mathbf{U}$-Statistics for Censored Data

Consider a triple $(Y,C,\mathbf{X})$ of random variables defined in $\mathbb{R}\times \mathbb{R}\times \mathcal{X}$. Here Y is the variable of interest, C is a censoring variable, and $\mathbf{X}$ is a concomitant variable. Throughout, we will use [96] notation and we work with a sample $\left\{{(Y_i,C_i,{\mathbf{X}}_i)}_{1\le i\le n}\right\}$ of independent and identically distributed replication of $(Y,C,\mathbf{X})$, $n\ge 1$. Actually, in the right censorship model, the pairs $(Y_i,C_i)$, $1\le i\le n$, are not directly observed, and the corresponding information is given by $Z_i:=min\{Y_i,C_i\}$ and ${\Delta }_i:=\mathbb{𝟙}\{Y_i\le C_i\}$, $1\le i\le n$. Accordingly, the observed sample is

$${\mathcal{D}}_n=\{(Z_i,{\Delta }_i,{\mathbf{X}}_i),i=1,\dots ,n\}.$$

This type of censoring is commonly applied to the survival data collected during clinical trials as well as the failure time data collected during reliability studies, for example. To be more specific, the majority of statistical experiments end up producing incomplete samples, even when the conditions are carefully monitored. For instance, clinical data for surviving the majority of diseases are typically censored due to the presence of other competing risks to life that ultimately result in death. In the sequel, we impose the following assumptions upon the distribution of $(\mathbf{X},Y)$. Denote by $\mathcal{I}$ a given compact set in $\mathcal{X}$ with nonempty interior and set, for any $\alpha >0$,

$${\mathcal{I}}_{\alpha }=\{\mathbf{x}:\underset{\mathbf{u}\in \mathcal{I}}{inf}\parallel \mathbf{x}-\mathbf{u}\parallel \le \alpha \}.$$

For $-\infty <t<\infty$, set

$$F_Y\left(t\right)=\mathbb{P}(Y\le t),G\left(t\right)=\mathbb{P}(C\le t),andH\left(t\right)=\mathbb{P}(Z\le t),$$

the right-continuous distribution functions of Y, C and Z, respectively. For any right-continuous distribution function L defined on $\mathbb{R}$, denote by

$$T_L=sup\{t\in \mathbb{R}:L\left(t\right)<1\}$$

the upper point of the corresponding distribution. Now, consider a pointwise measurable class $\mathcal{F}$ of real measurable functions defined on $\mathbb{R}$, and assume that $\mathcal{F}$ is of VC-type. We recall the regression function of $\psi \left(Y\right)$ evaluated at $\mathbf{X}=\mathbf{x}$, for $\psi \in \mathcal{F}$ and $\mathbf{x}\in {\mathcal{I}}_{\alpha }$, given by

$$r^{\left(1\right)}(\psi ,\mathbf{x})=\mathbb{E}(\psi \left(Y\right)\mid \mathbf{X}=\mathbf{x}),$$

when Y is right-censored. To estimate $r^{\left(1\right)}(\psi ,·)$, we make use of the Inverse Probability of Censoring Weighted (I.P.C.W.) estimators that have recently gained popularity in the censored data literature (see [97,98]). The key idea of I.P.C.W. estimators is as follows. Introduce the real-valued function ${\mathrm{\Phi }}_{\psi }(·,·)$ defined on ${\mathbb{R}}^2$ by

$${\mathrm{\Phi }}_{\psi }(y,c)=\frac{\mathbb{𝟙}\{y\le c\}\psi (y\wedge c)}{1-G(y\wedge c)}.$$

(17)

Assuming the function $G(·)$ to be known, first note that

$${\mathrm{\Phi }}_{\psi }(Y_i,C_i)={\Delta }_i\psi \left(Z_i\right)/(1-G\left(Z_i\right))$$

is observed for every $1\le i\le n$. In addition, under Assumption ($\mathbf{I}$) below

($\mathbf{I}$)

C and $(Y,\mathbf{X})$ are independent.

We have

$$\begin{array}{ccc}\hfill r^{\left(1\right)}({\mathrm{\Phi }}_{\psi },\mathbf{x})& :=& \mathbb{E}({\mathrm{\Phi }}_{\psi }(Y,C)\mid \mathbf{X}=\mathbf{x})\hfill \\ \hfill & =& \mathbb{E}\left\{\frac{\mathbb{𝟙}\{Y\le C\}\psi \left(Z\right)}{1-G\left(Z\right)}\mid \mathbf{X}=\mathbf{x}\right\}\hfill \\ \hfill & =& \mathbb{E}\left\{\frac{\psi \left(Y\right)}{1-G\left(Y\right)}\mathbb{E}(\mathbb{𝟙}\{Y\le C\}\mid \mathbf{X},Y)\mid \mathbf{X}=\mathbf{x}\right\}\hfill \\ & =& r^{\left(1\right)}(\psi ,\mathbf{x}).\hfill \end{array}$$

(18)

Therefore, every estimate of $r^{\left(1\right)}(\mathrm{\Phi }\psi ,·)$ that can be constructed using completely observed data is also an estimate of $r^{\left(1\right)}(\psi ,·)$. This characteristic permits the natural application of the majority of statistical procedures known to produce estimates of the regression function in the uncensored case to the censored case. Estimates of the kernel type, for instance, are exceptionally straightforward to build. Set, for $\mathbf{x}\in \mathcal{I}$, $h\ge 0$, $1\le i\le n$,

$$\begin{array}{c}\hfill {\overline{\omega }}_{n,K,h,i}^{\left(1\right)}\left(\mathbf{x}\right):={\delta }_{m_n}\left(\mathbf{x},{\mathbf{X}}_i\right)/\sum _{j=1}^n{\delta }_{m_n}\left(\mathbf{x},{\mathbf{X}}_j\right).\end{array}$$

(19)

Making use of the Equations (17)–(19), whenever $G(·)$ is known, we define the kernel estimator of $r^{\left(1\right)}(\psi ,·)$ by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(1\right)}(\psi ,\mathbf{x};h_n)=\sum _{i=1}^n{\overline{\omega }}_{n,K,h,i}^{\left(1\right)}\left(\mathbf{x}\right)\frac{{\Delta }_i\psi \left(Z_i\right)}{1-G\left(Z_i\right)}.\end{array}$$

(20)

Since the function $G(·)$ is unknown, it is to be estimated. Let $G_n^{*}(·)$ denote the Kaplan-Meier estimator of the function $G(·)$ [99]. To be precise, adopting the conventions

$$\prod _{\Phi }=1$$

and $0^0=1$ and setting

$$N_n\left(u\right)=\sum _{i=1}^n\mathbb{𝟙}\{Z_i\ge u\},$$

we have

$$G_n^{*}\left(u\right)=1-\prod _{i:Z_i\le u}{\left\{\frac{N_n\left(Z_i\right)-1}{N_n\left(Z_i\right)}\right\}}^{(1-{\Delta }_i)},\mathrm{f}\mathrm{o}\mathrm{r}u\in \mathbb{R}.$$

Given this notation, we will examine the next estimate of $r^{\left(1\right)}(\psi ,·)$

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(1\right)*}(\psi ,\mathbf{x};h_n)=\sum _{i=1}^n{\overline{\omega }}_{n,K,h,i}^{\left(1\right)}\left(\mathbf{x}\right)\frac{{\Delta }_i\psi \left(Z_i\right)}{1-G_n^{*}\left(Z_i\right)},\end{array}$$

(21)

the reader is invited to see the papers of [96,97]. The convention $0/0=0$ is used, this quantity is well defined, since $G_n^{*}\left(Z_i\right)=1$ if and only if $Z_i=Z_{\left(n\right)}$ and ${\Delta }_{\left(n\right)}=0$, where $Z_{\left(k\right)}$ is the kth ordered statistic related with the sample $(Z_1,\dots ,Z_n)$ for $k=1,\dots ,n$ and ${\Delta }_{\left(k\right)}$ is the ${\Delta }_j$ corresponding to $Z_k=Z_j$. When the variable of interest is right-censored, it is often impossible to estimate the function of the (conditional) law on the whole support (see [100]). Ref. [101] introduces a right-censored version of an unconditional U-statistic with a kernel of degree $m\ge 1$ based on the notion of a mean-preserving reweighting technique. Ref. [102] have demonstrated the almost sure convergence of multi-sample U-statistics under random censorship and presented an application by analyzing the consistency of a novel class of tests meant to evaluate distribution equality. Ref. [103] presented improvements to the traditional U-statistics to counteract potential biases caused by right-censoring of the outcomes and the existence of confounding factors. Ref. [104] suggested an alternative method for estimating the U-statistic by employing a substitution estimator of the conditional kernel given observed data. We also refer to [44,45,105]. To our best knowledge, estimating the conditional U-statistics in the censored data setting is a current open problem, and it gives the main motivation for the study of this section.

The function described by (17) has a natural expansion given by

$${\mathrm{\Phi }}_{\psi }(y_1,\dots ,y_k,c_1,\dots ,c_k)=\frac{{\prod }_{i=1}^k\{\mathbb{𝟙}\{y_i\le c_i\}\psi (y_1\wedge c_1,\dots ,y_k\wedge c_m)}{{\prod }_{i=1}^k\{1-G(y_i\wedge c_i)\}}.$$

We have an analogous relationship to (18) based on the formula:

$$\begin{array}{cc}{\displaystyle \mathbb{E}}\hfill & {\displaystyle ({\mathrm{\Phi }}_{\psi }(Y_1,\dots ,Y_k,C_1,\dots ,C_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{x})}\hfill \\ & =\mathbb{E}\left(\frac{{\prod }_{i=1}^k\{\mathbb{𝟙}\{Y_i\le C_i\}\psi (Y_1\wedge C_1,\dots ,Y_k\wedge C_k)}{{\prod }_{i=1}^k\{1-G(Y_i\wedge C_i)\}}\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{x}\right)\hfill \\ & =\mathbb{E}\left(\frac{\psi (Y_1,\dots ,Y_k)}{{\prod }_{i=1}^k\{1-G\left(Y_i\right)\}}\mathbb{E}\left(\prod _{i=1}^k\{\mathbb{𝟙}\{Y_i\le C_i\}\mid (Y_1,X_1),\dots (Y_k,X_k)\right)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{x}\right)\hfill \\ & =\mathbb{E}\left(\psi (Y_1,\dots ,Y_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{x}\right).\hfill \end{array}$$

An analogue estimator to (4) in the censored situation is given by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(k\right)}(\psi ,\mathbf{x};m_n)=\sum _{(i_1,\dots ,i_k)\in I(k,n)}\frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle (1-G\left(Z_{i_1}\right)\cdots (1-G\left(Z_{i_k}\right))}}{\overline{\omega }}_{n,\delta ,m_n,\mathbf{i}}^{\left(k\right)}\left(\mathbf{x}\right),\end{array}$$

(22)

where, for $\mathbf{i}=(i_1,\dots ,i_k)\in I(k,n)$,

$$\begin{array}{c}\hfill {\overline{\omega }}_{n,\delta ,m_n,\mathbf{i}}^{\left(k\right)}\left(\mathbf{x}\right)=\frac{{\delta }_{m_n}\left({\mathbf{x}}_1,{\mathbf{X}}_{i_1}\right)\cdots {\delta }_{m_n}\left({\mathbf{x}}_k,{\mathbf{X}}_{i_k}\right)}{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}{\delta }_{m_n}\left({\mathbf{x}}_1,{\mathbf{X}}_{i_1}\right)\cdots {\delta }_{m_n}\left({\mathbf{x}}_k,{\mathbf{X}}_{i_k}\right)}}.\end{array}$$

(23)

The estimator we shall examine is provided by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(k\right)*}(\psi ,\mathbf{x};m_n)=\sum _{(i_1,\dots ,i_k)\in I(k,n)}\frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle (1-G_n^{*}\left(Z_{i_1}\right)\cdots (1-G_n^{*}\left(Z_{i_k}\right))}}{\overline{\omega }}_{n,\delta ,m_n,\mathbf{i}}^{\left(k\right)}\left(\mathbf{x}\right).\end{array}$$

(24)

In a similar way as in [44], we arrive to the following conclusion.

Corollary 3. 

Assume that the condition ($\mathbf{I}$) and the assumptions of Theorems 3 and 5 are satisfied. Then, we have

$$\left|{\breve{r}}_n^{\left(k\right)*}(\psi ,\mathbf{x};m_n)-r^{\left(k\right)}(\phi ,\tilde{\mathbf{x}})\right|=O\left(D_m\right)+O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right),a.s.$$

This last result is a direct consequence of Corollary (2) and the law of iterated logarithm for $G_n^{*}(·)$ obtained in [106] gives

$$\underset{t\le \tau }{sup}|G_n^{*}-G\left(t\right)|=O\left(\sqrt{\frac{loglogn}{n}}\right)\mathrm{almost}\mathrm{surely}\mathrm{as}n\to \infty .$$

At this point, we may refer to [44,45,105].

5. Applications

5.1. Kendall Rank Correlation Coefficient

To test the independence of one-dimensional random variables $Y_1$ and $Y_2$ [107] proposed a method based on the U-statistic $K_n$ with the kernel function:

$$\phi \left(\left(s_1,t_1\right),\left(s_2,t_2\right)\right)={\mathbb{𝟙}}_{\left\{\left(s_2-s_1\right)\left(t_2-t_1\right)>0\right\}}-{\mathbb{𝟙}}_{\left\{\left(s_2-s_1\right)\left(t_2-t_1\right)⩽0\right\}·}$$

(25)

Its rejection on the region is of the form $\left\{\sqrt{n}{K}_n>\gamma \right\}$. In this example, we consider a multivariate case. To test the conditional independence of $\mathit{\xi },\mathit{\eta }:Y=(\mathit{\xi },\mathit{\eta })$ given X, we propose a method based on the conditional U-statistic:

$${\widehat{r}}_n^{\left(2\right)}(\phi ,\mathbf{t})=\frac{{\sum }_{i\ne j}^n\phi \left(Y_i,Y_j\right){\delta }_m(t_1,X_i){\delta }_m(t_2,X_j)}{{\sum }_{i\ne j}^n{\delta }_m(t_1,X_i){\delta }_m(t_2,X_j)},$$

where $\mathbf{t}=\left(t_1,t_2\right)\in \mathbb{I}\subset {\mathbb{R}}^2$ and $\phi (·)$ is Kendall’s kernel (25). Suppose that $\xi$ and $\eta$ are $d_1$ and $d_2$-dimensional random vectors, respectively, and $d_1+d_2=d$. Furthermore, suppose that ${\mathrm{Y}}_1,\dots ,{\mathrm{Y}}_n$ are observations of $(\xi ,\eta )$, we are interested in testing:

$${\mathrm{H}}_0:\xi \mathrm{a}\mathrm{n}\mathrm{d}\eta \text{are} \text{conditionally} \text{independent} \text{given}X.\mathrm{v}\mathrm{s}{\mathrm{H}}_a:{\mathrm{H}}_0\text{is} \text{not} \text{true}.$$

(26)

Let $\mathbf{a}=\left({\mathbf{a}}_1,{\mathbf{a}}_2\right)\in {\mathbb{R}}^d$ such as $\parallel \mathbf{a}\parallel =1$ and ${\mathbf{a}}_1\in {\mathbb{R}}^{d_1},{\mathbf{a}}_2\in {\mathbb{R}}^{d_2}$, and $\mathrm{F}(·),\mathrm{G}(·)$ be the distribution functions of $\xi$ and $\eta$, respectively. Suppose $F^{a_1}(·)$ and $G^{a_2}(·)$ to be continuous for any unit vector $\mathbf{a}=\left({\mathbf{a}}_1,{\mathbf{a}}_2\right)$ where ${\mathrm{F}}^{{\mathbf{a}}_1}\left(t\right)=\mathbb{P}\left({\mathbf{a}}_1^{\top }\xi <t\right)$ and ${\mathrm{G}}^{{\mathbf{a}}_2}\left(t\right)=\mathbb{P}\left({\mathbf{a}}_2^{\top }\eta <t\right)$ and ${\mathbf{a}}_1^{\mathrm{T}}$ means the transpose of the vector ${\mathbf{a}}_i,1⩽i⩽2$. For $n=2$, let $Y^{\left(1\right)}=\left({\xi }^{\left(1\right)},{\eta }^{\left(1\right)}\right)$ and $Y^{\left(2\right)}=\left({\xi }^{\left(2\right)},{\eta }^{\left(2\right)}\right)$ such as ${\xi }^{\left(i\right)}\in {\mathbb{R}}^{d_1}$ and ${\eta }^{\left(i\right)}\in {\mathbb{R}}^{d_2}$ for $i=1,2$, and:

$${\phi }^a\left({\mathrm{Y}}^{\left(1\right)},{\mathrm{Y}}^{\left(2\right)}\right)=\phi \left(\left({\mathbf{a}}_1^{\top }{\xi }^{\left(1\right)},{\mathbf{a}}_2^{\top }{\eta }^{\left(1\right)}\right),\left({\mathbf{a}}_1^{\top }{\xi }^{\left(2\right)},{\mathbf{a}}_2^{\top }{\eta }^{\left(2\right)}\right)\right).$$

An application of Corollary 2 gives

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^2}{sup}\left|{\widehat{r}}_n^{\left(2\right)}({\phi }^a,\mathbf{x};m_n)-r^{\left(2\right)}({\phi }^a,\mathbf{x})\right|=O\left(D_m\right)+O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

(27)

5.2. Discrimination Problems

Now, we apply these findings to the discrimination problem outlined in Section 3 of [108], refer to also to [109]. We will employ a similar setup and notation. Let $\phi (·)$ be any function taking at most finitely many values, say $1,\dots ,M$. The sets

$$A_j=\left\{({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k):\phi ({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k)=j\right\},1\le j\le M$$

subsequently, produce a partition of the feature space. Predicting the value of $\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)$ is equivalent to making a guess about which set will be in the partition to which $({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)$ belongs. For any discrimination rule g, we have

$$\mathbb{P}(g\left(\mathbf{X}\right)=\phi \left(\mathbf{Y}\right))\le \sum _{j=1}^M{\int }_{\tilde{\mathbf{x}}:g\left(\tilde{\mathbf{x}}\right)=j\}}max{\mathfrak{M}}^j\left(\tilde{\mathbf{x}}\right)d\mathbb{P}\left(\tilde{\mathbf{x}}\right),$$

where

$${\mathfrak{M}}^j\left(\tilde{\mathbf{x}}\right)=\mathbb{P}(\phi \left(\mathbf{Y}\right)=j\mid \mathbf{X}=\tilde{\mathbf{x}}),\tilde{\mathbf{x}}\in {\mathbb{R}}^d.$$

The inequality described above becomes equality if

$$g_0\left(\tilde{\mathbf{x}}\right)=arg\underset{1\le j\le M}{max}{\mathfrak{M}}^j\left(\tilde{\mathbf{x}}\right).$$

$g_0(·)$ is known as the Bayes rule, and the associated error probability

$${\mathbf{L}}^{*}=1-\mathbb{P}(g_0\left(\mathbf{X}\right)=\phi \left(\mathbf{Y}\right))=1-\mathbb{E}\left\{\underset{1\le j\le M}{max}{\mathfrak{M}}^j\left(\tilde{\mathbf{x}}\right)\right\}$$

is called the Bayes risk. Each of the unknown ${\mathfrak{M}}^j$ functions can be reliably estimated using one of the techniques described in the prior sections. Let, for $1\le j\le M$,

$$\begin{array}{c}\hfill {\mathfrak{M}}_n^j\left(\tilde{\mathbf{x}}\right)=\frac{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}\mathbb{𝟙}\{\phi (Y_{i_1},\dots ,Y_{i_k})=j\}{\delta }_{m_n}\left({\mathbf{x}}_1,{\mathbf{X}}_{i_1}\right)\cdots {\delta }_{m_n}\left({\mathbf{x}}_k,{\mathbf{X}}_{i_k}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}{\delta }_{m_n}\left({\mathbf{x}}_1,{\mathbf{X}}_{i_1}\right)\cdots {\delta }_{m_n}\left({\mathbf{x}}_k,{\mathbf{X}}_{i_k}\right)}},\end{array}$$

(28)

Set

$$g_{0,n}\left(\tilde{\mathbf{x}}\right)=arg\underset{1\le j\le M}{max}{\mathfrak{M}}_n^j\left(\tilde{\mathbf{x}}\right).$$

Let us define

$${\mathbf{L}}_n^{*}=\mathbb{P}(g_{0,n}\left(\mathbf{X}\right)\ne \phi \left(\mathbf{Y}\right)).$$

The discrimination rule $g_{0,n}(·)$ is asymptotically Bayes’ risk consistent

$${\mathbf{L}}_n^{*}\to {\mathbf{L}}^{*}.$$

This is a consequence of the relation

$$\left|{\mathbf{L}}^{*}-{\mathbf{L}}_n^{*}\right|\le 2\mathbb{E}\left[\underset{1\le j\le M}{max}\left|{\mathfrak{M}}_n^j\left(\mathit{X}\right)-{\mathfrak{M}}^j\left(\mathit{X}\right)\right|\right].$$

5.3. Metric Learning

Metric learning seeks to adapt the metric to the data and has garnered a great deal of attention in recent years; for instance, see [110,111] for an account of metric learning and its applications. This is driven by a wide variety of applications, spanning from information retrieval via bioinformatics to computer vision as the primary source of inspiration. For the purpose of demonstrating the applicability of this idea, we will now discuss the metric learning problem for supervised classification as shown in [111]. Let us consider independent copies $\left(X_1,Y_1\right),\dots ,\left(X_n,Y_n\right)$ of a $\mathcal{X}\times \mathcal{Y}$ valued random couple $(X,Y)$, where $\mathcal{X}$ is some feature space and $\mathcal{Y}=\{1,\dots ,C\}$, with $C\ge 2$ say, a finite set of labels. Let $\mathcal{D}$ be a set distance measures $D:\mathcal{X}\times \mathcal{X}\to {\mathbb{R}}_{+}$. The purpose of metric learning in this context is, intuitively speaking, to identify a metric under which pairs of points with the same label are close to each other, while those with different labels are far away from each other. A natural way to characterize the risk associated with a metric D is as follows

$$R\left(D\right)=\mathbb{E}\left[\varphi \left(\left(1-D\left(X,X^{\prime }\right)·\left(2\mathbb{𝟙}\left\{Y=Y^{\prime }\right\}-1\right)\right)\right],\right.$$

(29)

where $\varphi \left(u\right)$ is a convex loss function upper bounding the indicator function $\mathbb{𝟙}\{u\ge 0\}$, for instance, the hinge loss $\varphi \left(u\right)=max(0,1-u)$. To estimate $R\left(D\right)$, we consider the usual empirical estimator

$$R_n\left(D\right)=\frac{2}{n(n-1)}\sum _{1\le i<j\le n}\varphi \left(\left(D\left(X_i,X_j\right)-1\right)·\left(2\mathbb{𝟙}\left\{Y_i=Y_j\right\}-1\right)\right),$$

(30)

which is one sample U-statistic of degree two with kernel given by:

$${\phi }_{\mathrm{D}}\left((x,y),\left(x^{\prime },y^{\prime }\right)\right)=\varphi \left(\left(D\left(x,x^{\prime }\right)-1\right)·\left(2\mathbb{𝟙}\left\{y=y^{\prime }\right\}-1\right)\right).$$

The convergence to (29) of a minimizer of (30) has been studied in the frameworks of algorithmic stability ([112]), algorithmic robustness ([113]) and based on the theory of U-processes under appropriate regularization ([114]).

5.4. Time Series Prediction from Continuous Set of Past Values

Let ${\{Z_n\left(t\right),t\in \mathbb{R}\}}_{n\ge 1}$ denote a sequence of processes with value in $\mathbb{R}$. Let s denote a fixed positive real number. In this model, we suppose that the process is observed from $t=0$ until $t=t_{max}$, and assume without loss of generality that $t_{max}=nT+s<\tau$. The method ensures splitting the observed process into n fixed-length segments. Let us denote each piece of the process by

$$X_i=\{Z\left(t\right),(i-1)T\le t<iT\}.$$

The response value is therefore $Y_i=Z(iT+s)$, and this can be formulated as a regression problem:

$$\phi (Z_1(\tau +s),\dots ,Z_k(\tau +s))=r^{\left(k\right)}(Z_1\left(t\right),\dots ,Z_k\left(t\right)),\mathrm{f}\mathrm{o}\mathrm{r}\tau -T\le t<\tau .$$

(31)

provided that we make the assumption that a function of this kind, r, does not depend on i (which is satisfied if the process is stationary, for example). Because of this, when we get to time $\tau$, we can use the following predictor, which is directly derived from our estimator, to make a prediction about the value that will be at time $\tau +s$

$${\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{z};m_n)=\frac{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}\phi (Z_{i_1}(\tau +s),\dots ,Z_{i_k}(\tau +s)){\delta }_{m_n}\left(z_1,X_{i_1}\right)\cdots {\delta }_{m_n}\left(z_k,X_{i_k}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_k)\in I(k,n)}{\delta }_{m_n}\left(z_1,X_{i_1}\right)\cdots {\delta }_{m_n}\left(z_k,X_{i_k}\right)}}$$

where $\mathbf{z}=(z_1,\dots ,z_k)=\{(Z_1\left(t\right),\dots ,Z_k\left(t\right))$, for $\tau -T\le t<\tau \}$. Corollary 2 provides mathematical support for this nonparametric functional predictor and extends previous results in numerous ways in [48,78]. Notice that this modelization encompasses a wide variety of practical applications, as this procedure allows for the consideration of a large number of past process values without being affected by the curse of dimensionality.

5.5. Example of U-Kernels

Example 4. 

Hoeffding’s D From the symmetric kernel,

$$\begin{array}{ccc}& & h_D\left(z_1,\dots ,z_5\right)\hfill \\ & :=& \frac{1}{16}\sum _{\left(i_1,\dots ,i_5\right)\in {\mathcal{P}}_5}\left[\left\{\mathbf{1}\left(z_{i_1,1}\le z_{i_5,1}\right)-\mathbf{1}\left(z_{i_2,1}\le z_{i_5,1}\right)\right\}\left\{\mathbf{1}\left(z_{i_3,1}\le z_{i_5,1}\right)-\mathbf{1}\left(z_{i_4,1}\le z_{i_5,1}\right)\right\}\right]\hfill \\ & & \times \left[\left\{\mathbf{1}\left(z_{i_1,2}\le z_{i_5,2}\right)-\mathbf{1}\left(z_{i_2,2}\le z_{i_5,2}\right)\right\}\left\{\mathbf{1}\left(z_{i_3,2}\le z_{i_5,2}\right)-\mathbf{1}\left(z_{i_4,2}\le z_{i_5,2}\right)\right\}\right].\hfill \end{array}$$

We obtain Hoeffding’s D statistic, which is a rank-based U-statistic of order 5.

Example 5 

(Blum-Kiefer-Rosenblatt’s R). The symmetric kernel

$$\begin{array}{ccc}& & h_R\left(z_1,\dots ,z_6\right)\hfill \\ & :=& \frac{1}{32}\sum _{\left(i_1,\dots ,i_6\right)\in {\mathcal{P}}_6}\left[\left\{\mathbf{1}\left(z_{i_1,1}\le z_{i_5,1}\right)-\mathbf{1}\left(z_{i_2,1}\le z_{i_5,1}\right)\right\}\left\{\mathbf{1}\left(z_{i_3,1}\le z_{i_5,1}\right)-\mathbf{1}\left(z_{i_4,1}\le z_{i_5,1}\right)\right\}\right]\hfill \\ & & \times \left[\left\{\mathbf{1}\left(z_{i_1,2}\le z_{i_6,2}\right)-\mathbf{1}\left(z_{i_2,2}\le z_{i_6,2}\right)\right\}\left\{\mathbf{1}\left(z_{i_3,2}\le z_{i_6,2}\right)-\mathbf{1}\left(z_{i_4,2}\le z_{i_6,2}\right)\right\}\right],\hfill \end{array}$$

gives Blum-Kiefer-Rosenblatt’s R statistic (see [115]), which is a rank-based U-statistic of order 6, refer also to [116,117,118,119,120].

Example 6. 

Bergsma-Dassios-Yanagimoto’s ${\tau }^{*}$ [121] introduced a rank correlation statistic as a U-statistic of order 4 with the symmetric kernel

$$\begin{array}{c}{h}_{{\tau }^{*}}{\displaystyle \left(z_1\right.\left.,\dots ,z_4\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}:=\frac{1}{16}\sum _{\left(i_1,\dots ,i_4\right)\in {\mathcal{P}}_4}\left\{\mathbf{1}\left(z_{i_1,1},z_{i_3,1}<z_{i_2,1},z_{i_4,1}\right)+\mathbf{1}\left(z_{i_2,1},z_{i_4,1}<z_{i_1,1},z_{i_3,1}\right)\right.\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left.-\mathbf{1}\left(z_{i_1,1},z_{i_4,1}<z_{i_2,1},z_{i_3,1}\right)-\mathbf{1}\left(z_{i_2,1},z_{i_3,1}<z_{i_1,1},z_{i_4,1}\right)\right\}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\times \left\{\mathbf{1}\left(z_{i_1,2},z_{i_3,2}<z_{i_2,2},z_{i_4,2}\right)+\mathbf{1}\left(z_{i_2,2},z_{i_4,2}<z_{i_1,2},z_{i_3,2}\right)\right.\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left.-\mathbf{1}\left(z_{i_1,2},z_{i_4,2}<z_{i_2,2},z_{i_3,2}\right)-\mathbf{1}\left(z_{i_2,2},z_{i_3,2}<z_{i_1,2},z_{i_4,2}\right)\right\}.\hfill \end{array}$$

Here

$$\mathbf{1}\left(y_1,y_2<y_3,y_4\right):=\mathbf{1}\left(y_1<y_3\right)\mathbf{1}\left(y_1<y_4\right)\mathbf{1}\left(y_2<y_3\right)\mathbf{1}\left(y_2<y_4\right).$$

Example 7. 

Two generic vectors $y=\left(y_1,y_2\right)$ and $z=\left(z_1,z_2\right)$ in ${\mathbb{R}}^2$ are said to be concordant if $\left(y_1-z_1\right)\left(y_2-z_2\right)>0$. For $m,k=1,\dots ,p$, define

$${\tau }_{mk}=\frac{1}{n(n-1)}\sum _{1\le i\ne j\le n}\mathbf{1}\left\{\left(X_{im}-X_{jm}\right)\left(X_{ik}-X_{jk}\right)>0\right\}.$$

Then, Kendall’s tau rank correlation coefficient matrix $T={\left\{{\tau }_{mk}\right\}}_{m,k=1}^p$ denotes a matrix-valued U-statistic, for wich the kernel is bounded. It is obvious that ${\tau }_{mk}$ quantifies the monotonic dependency between $\left(X_{1m},X_{1k}\right)$ and $\left(X_{2m},X_{2k}\right)$ and it is an unbiased estimator of

$$\mathbb{P}\left(\left(X_{1m}-X_{2m}\right)\left(X_{1k}-X_{2k}\right)>0\right),$$

that is, the probability that $\left(X_{1m},X_{1k}\right)$ and $\left(X_{2m},X_{2k}\right)$ are concordant.

Example 8. 

The Gini mean difference. The Gini index provides another usual measure of dispersion. It corresponds to the case where $E\subset \mathbb{R}$ and $h(x,y)=|x-y|$:

$$G_n=\frac{2}{n(n-1)}\sum _{1\le i<j\le n}\left|X_i-X_j\right|$$

6. Concluding Remarks

In this paper, the conditional U-statistics regression operator estimation methods for random elements taking values in an infinite-dimensional separable Banach space are generalized to the delta-sequences techniques. The space of continuous functions on the interval $(0,1)$ with the supremum norm illustrates a separable Banach space. Notably, the method of delta-sequences unifies the kernel method of the probability density function estimation, the histogram method, and a few other methods, including the method of orthogonal series for appropriate choices of orthonormal bases in the one-dimensional and finite-dimensional cases. We have obtained strong uniform consistency results in abstract settings under some conditions on the model. The general framework that we consider extends the existing methods to higher-order statistics; this has a significant impact both from a theoretical and practical point of view. In a future investigation, considering the limiting law of the conditional U-statistics regression estimators based on the delta sequence will be of interest. A natural extension of the present investigation is to consider the serial-dependent setting such as the mixing (see [61,62,122]) or the ergodic processes (see [56,123]). In a future investigation of the functional delta sequence local linear approach estimators, it will be natural to think about the possibility of obtaining an alternative estimator that benefits from the advantages of both methods, the local linear method and the delta sequence approach. This is because both methods have their own distinct advantages. Many methods have been developed and established to construct, in asymptotically optimal ways, bandwidth selection rules for nonparametric kernel estimators, particularly for the Nadaraya-Watson regression estimator. We quote several of these methods, including [44,45,124]. This parameter needs to have an appropriate value chosen for it to ensure that satisfactory practical performances are achieved, either in the typical situation of finite dimensions or in the framework of infinite dimensions. On the other hand, to the best of our knowledge, no such studies are currently conducted to treat generic functional conditional U-statistics. This exemplifies a potential new avenue for research in the future.

7. Mathematical Development

This section contains the proof of our results. The preceding notation is also used in the subsequent text. Keeping in mind the relation (7), we can conclude that, for each $\mathbf{x}=(x_1,\dots ,x_k)\in S_{\mathcal{X}}^k$, there exists $\ell \left(\mathbf{x}\right)=(\ell \left(x_1\right),\dots ,\ell \left(x_k\right))$ where $\forall 1\le i\le k$, $1\le \ell \left(x_i\right)\le d_n$ and such that

$$\mathbf{x}\in \prod _{i=1}^{k}B(x_{\ell \left(x_i\right)},{\epsilon }_n)andd(x_i,x_{\ell \left(x_i\right)})=\underset{1\le \ell \le d_n}{argmin}d(x_i,x_{\ell }).$$

We denote for each $\mathbf{x}=(x_1,\dots ,x_k)\in S_{\mathcal{X}}^k$ and ${\mathbf{x}}_{\ell \left(\mathbf{x}\right)}=(x_{\ell \left(x_1\right)},\dots ,x_{\ell \left(x_k\right)})$:

$$\prod _{i=1}^{k}B(x_{\ell \left(x_i\right)},{\epsilon }_n)=:B({\mathbf{x}}_{\ell \left(\mathbf{x}\right)},{\epsilon }_n).$$

Hence, for each $\mathbf{x}\in S_{\mathcal{X}}^k$, we can reformulated the U-statistic as

$$\begin{array}{c}{\displaystyle \left|u_n(\phi ,\mathbf{x};m_n)-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|}\hfill \\ \le \left|u_n(\phi ,\mathbf{x};m_n)-u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right|\hfill \\ +\left|\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|\hfill \\ +\left|u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]\right|.\hfill \end{array}$$

Proof of Theorem 1. 

We need to establish that there exists some $\eta >0$, in such a way that

$$\sum _{n\ge 1}\mathbb{P}\left\{\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\sqrt{\frac{m}{s_{m}log\left(m\right)}}\left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)\right|\ge \eta \right\}<\infty .$$

(32)

To do that, we need to obtain an exponential bound for

$$\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left[u_n(\phi ,\mathbf{x},m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

We first remark that we have

$$\begin{array}{cc}\hfill & \left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left[u_n(\phi ,\mathbf{x},m_n)\right]\right|\hfill \\ \hfill & =\frac{(n-k)!}{n!}\left|\sum _{i\in I(k,n)}\left\{\phi (Y_{i_1},\dots ,Y_{i_k})\prod _{j=1}^k{\delta }_{m_n}(x_j,X_{i_j})-\mathbb{E}\left[\phi (Y_{i_1},\dots ,Y_{i_k})\prod _{j=1}^k{\delta }_{m_n}(x_j,X_{i_j})\right]\right\}\right|\hfill \\ \hfill & =\frac{(n-k)!}{n!}\left|\sum _{i\in I(k,n)}\left\{G_{\phi ,\mathbf{x}}({\mathbf{X}}_i,{\mathbf{Y}}_i)-\mathbb{E}\left[G_{\phi ,\mathbf{x}}({\mathbf{X}}_i,{\mathbf{Y}}_i)\right]\right\}\right|\hfill \\ \hfill & =\frac{(n-k)!}{n!}\left|\sum _{i\in I(k,n)}H({\mathbf{X}}_i,{\mathbf{Y}}_i)\right|,\hfill \end{array}$$

where

$$H(\mathbf{X},\mathbf{Y})=G_{\phi ,\mathbf{x}}(\mathbf{X},\mathbf{Y})-\mathbb{E}\left[G_{\phi ,\mathbf{x}}(\mathbf{X},\mathbf{Y})\right].$$

In order to get the desired result, we apply Lemma A1 on the function $H(·,·)$. Throughout the rest of the proof, we suppose the function $G_{\phi ,\mathbf{x}}$ is symmetric. Moreover, it is clear that the function $H(·,·)$ is bounded by $2M{C}_1{s}_m$ by condition (C.2) and the fact that the function $\phi (·)$ is bounded by the condition (C.7). We obviously remark that,

$$\theta =\mathbb{E}\left[H\right(\mathbf{X},\mathbf{Y}\left)\right]=0$$

by design, and

$$\begin{array}{cc}\hfill {\sigma }^2& =Var\left(H(\mathbf{X},\mathbf{Y})\right)\le 2{\left(M{C}_1{s}_m\right)}^2.\hfill \end{array}$$

For any $\eta >0$ and m large enough, we obtain that

$$\begin{array}{cc}\hfill & \mathbb{P}\left(\left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left[u_n(\phi ,\mathbf{x},m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hfill & \le 2exp\left[-\frac{n((s_{m}log\left(m\right))/m){\eta }^2}{4{\left(M{C}_1{s}_m\right)}^2+\frac{4}{3}M{C}_1{s}_m\eta \sqrt{(s_{m}log\left(m\right))/m}}\right].\hfill \end{array}$$

(33)

We can write

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x};m_n)-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|>2\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \le \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x};m_n)-u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right.\right.\hfill \end{array}$$

$$\begin{array}{ccc}& & \left.\left.+\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \end{array}$$

(34)

$$\begin{array}{ccc}& & +\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).\hfill \end{array}$$

(35)

Taking into account the condition (C.3), we have

$$\begin{array}{cc}\hfill & \left|u_n(\phi ,\mathbf{x};m_n)-u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}\left|\phi (Y_{i_1},\dots ,Y_{i_k})\left\{\prod _{j=1}^k{\delta }_{m_n}(x_j,X_{i_j})-\prod _{j=1}^k{\delta }_{m_n}(x_{\ell \left(x_j\right)},X_{i_j})\right\}\right|\hfill \\ \hfill & \le M\frac{(n-k)!}{n!}\sum _{i\in I(k,n)}\left|{\delta }_{m_n}(\mathbf{x},{\mathbf{X}}_i)-{\delta }_{m_n}({\mathbf{x}}_{\ell \left(\mathbf{x}\right)},{\mathbf{X}}_i)\right|\hfill \\ \hfill & \le M\frac{(n-k)!}{n!}\sum _{i\in I(k,n)}{C}_2{s}_m^{{\beta }_2}d{(\mathbf{x},{\mathbf{x}}_{\ell \left(\mathbf{x}\right)})}^{{\beta }_1}\hfill \\ \hfill & \le M{C}_2{s}_m^{{\beta }_2}d{(\mathbf{x},{\mathbf{x}}_{\ell \left(\mathbf{x}\right)})}^{{\beta }_1}\hfill \\ \hfill & \le M{C}_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}.\hfill \end{array}$$

Consequently, we obtain uniformly on $\mathbf{x}\in S_{\mathcal{X}}^k$:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x};m_n)-u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right|\le O\left(s_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\right)=O\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right),$$

(36)

by condition (C.5). We deduce from (36) that:

$$\begin{array}{c}{\displaystyle \left|\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|}\hfill \\ =\left|\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-u_n(\phi ,\mathbf{x};m_n)\right]\right|\hfill \\ \le \mathbb{E}\left|\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-u_n(\phi ,\mathbf{x};m_n)\right]\right|.\hfill \end{array}$$

(37)

The passage from (37) to (37) follows by applying Jensen’s inequality further to some properties of the absolute value function. Now using the fact that the function $\phi (·)$ is bounded and that the function ${\delta }_m$ is Lipschitz in addition for any constant a, $\mathbb{E}\left[a\right]=a$, we can directly conclude that

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|\le O\left(s_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\right)=O\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

For some $\eta >0$ and for sufficiently large n and large m, we have

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x};m_n)-u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right.\right.}\hfill \\ \left.\left.+\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)=0.\hfill \end{array}$$

Continue, now, with (35), supposing that the kernel function $G_{\phi ,{\mathbf{x}}_{\ell }}(·)$ is symmetric, we have to decompose the U-statistic by making use of the [8] decomposition, we infer that

$$\begin{array}{c}{\displaystyle u_n(\phi ,{\mathbf{x}}_{\ell };m_n)-\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell };m_n)\right]}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\sum _{p=1}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell },m_n}\right)\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=k{u}_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)+\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right).\hfill \end{array}$$

(38)

Let us first start with the linear term. We have

$$k{u}_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)=\frac{k}{n}\sum _{j=1}^n{\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)({\mathbf{X}}_i,{\mathbf{Y}}_i).$$

From Hoeffding’s projection, we have

$$\begin{array}{c}{\displaystyle {\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)(x,y)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\left\{\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}\left((x,X_2,\dots ,X_k),(y,Y_2,\dots ,Y_k)\right)\right]-\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}\left(\mathbf{X},\mathbf{Y}\right)\right]\right\}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\left\{\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}\left(\mathbf{X},\mathbf{Y}\right)\right|(X_1,Y_1)=(x,y)]-\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}\left(\mathbf{X},\mathbf{Y}\right)\right]\right\}.\hfill \end{array}$$

Set

$$Z_i={\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)(X_i,Y_i).$$

We can see that $Z_i$ are independent and identically distributed random variables bounded by $2kM{C}_1{s}_m$ with zero mean and

$${\sigma }^2\le {\left(M{C}_1{s}_m\right)}^2.$$

An application of Bernstein’s inequality yields

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \sum _{i=1}^{d_n}\mathbb{P}\left(\underset{1\le {\ell }_i\le d_n}{max}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2d_{n}exp\left[-\frac{n((s_{m}log\left(m\right))/m){\eta }^2}{4{\left(M{C}_1{s}_m\right)}^2+\frac{4}{3}M{C}_1{s}_m\eta \sqrt{(s_{m}log\left(m\right))/m}}\right]\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le n^{\alpha -\tau {\eta }^2/C_4},\hfill \end{array}$$

resulting from the fact $m\le n$ and $log\left(m\right)\ge \tau log\left(n\right)$. This implies that

$$\begin{array}{cc}\hfill & \sum _{n\ge 1}\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hfill & \le \sum _{n\ge 1}{n}^{\alpha -\tau {\eta }^2/C_4}<\infty .\hfill \end{array}$$

Consequently, we obtain the following:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

Moving to the nonlinear term, we want to prove that for $2\le p\le k$:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\left(\begin{array}{c}k\\ p\end{array}\right)\sqrt{m}\left|u_n^{\left(p\right)}\left({\pi }_{p,k}{\mathrm{G}}_{\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)}}\right)\right|}{\sqrt{s_{m}log\left(m\right)}}={\mathrm{O}}_{a.co}\left(1\right),$$

which implies that, for $1\le i\le k$ and $\ell =({\ell }_1,\dots ,{\ell }_k)$:

$$\underset{1\le {\ell }_i\le d_n}{max}\frac{\left(\begin{array}{c}k\\ p\end{array}\right)\sqrt{m}\left|u_n^{\left(p\right)}\left({\pi }_{p,k}{\mathrm{G}}_{\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)}}\right)\right|}{\sqrt{s_{m}log\left(m\right)}}={\mathrm{O}}_{a.co}\left(1\right).$$

To prove the above-mentioned equation, we need to apply Proposition 1 of [125] (see Lemma A2). We can see that $G_{\phi ,{\mathbf{x}}_{\ell }}$ is bounded by $M{C}_1{s}_m$, hence for $\eta >0$ we have

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(n^{1/2}\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\mathbb{P}\left(\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>n^{-1/2}\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\mathbb{P}\left(\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right),\hfill \end{array}$$

where ${\displaystyle {\epsilon }_0=\frac{\eta }{\sqrt{n}}}$. Now for ${\displaystyle t=\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}}$, Lemma A2 gives us:

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2exp\left(-\frac{t{(n-1)}^{1/2}}{2^{k+2}{k}^{k+1}M{C}_1{s}_m}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2exp\left(-\frac{\eta \sqrt{s_{m}log\left(m\right)/m}{(n-1)}^{1/2}}{2^{k+2}{k}^{k+1}M{C}_1{s}_m}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2exp\left(-\frac{\eta \sqrt{log\left(m\right)/m}{(n-1)}^{1/2}}{2^{k+2}{k}^{k+1}M{C}_1\sqrt{s_m}}\right).\hfill \end{array}$$

By the fact that $m\le n$ and $log\left(m\right)\ge \tau log\left(n\right)$, it follows that there exists $\eta >0$ in such a way that

$$\begin{array}{c}\hfill \mathbb{P}\left(\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\le n^{-\tau /2C_5},\end{array}$$

where $C_5=C^{″}{2}^{k+2}{k}^{k+1}M{C}_1\sqrt{s_m}$ with $C^{″}>0.$ Therefore, for each ${\epsilon }_0>0$, $1\le i\le k$ and $\ell =({\ell }_1,\dots ,{\ell }_k)$:

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le d_n\underset{1\le {\ell }_i\le d_n}{max}\mathbb{P}\left(\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le n^{-k(\tau /2C_5)}.\hfill \end{array}$$

Consequently, we have

$$\begin{array}{c}{\displaystyle \sum _{n\ge 1}\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\le \sum _{n\ge 1}{n}^{\alpha -\tau /2C_5}\to 0\mathrm{a}\mathrm{s}n\to 0.\hfill \end{array}$$

Hence, the proof is achieved. □

Proof of Theorem 2. 

We will need to truncate the conditional U-statistic to prove this theorem. Taking the condition $(\mathbf{C}.{\mathbf{7}}^{\prime })$ into account, for each $\lambda >0$ and

$${\displaystyle {\xi }_n:={\xi }_{m_n}=\frac{m_n}{log{m}_n}=:\frac{m}{logm},}$$

we have

$$\begin{array}{cc}\hfill G_{\phi ,\mathbf{x}}(\mathbf{x},\mathbf{y})& =G_{\phi ,\mathbf{x}}(\mathbf{x},\mathbf{y}){\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)\le \lambda {\xi }_n^{1/q}\}}+G_{\phi ,\mathbf{x}}(\mathbf{x},\mathbf{y}){\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\hfill \\ \hfill & =:G_{\phi ,\mathbf{x}}^{\left(T\right)}(\mathbf{x},\mathbf{y})+G_{\phi ,\mathbf{x}}^{\left(R\right)}(\mathbf{x},\mathbf{y}),\hfill \end{array}$$

which means that each function $\phi (·)$ is truncated as follows:

$$\begin{array}{ccc}\hfill \phi \left(\mathbf{y}\right)& =& \phi \left(\mathbf{y}\right){\mathbb{𝟙}}_{\left\{\phi \left(\mathbf{y}\right)\le \lambda {\xi }_n^{1/q}\right\}}+\phi \left(\mathbf{y}\right){\mathbb{𝟙}}_{\left\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\right\}}\hfill \\ & =& {\phi }^{\left(T\right)}\left(\mathbf{y}\right)+{\phi }^{\left(R\right)}\left(\mathbf{y}\right).\hfill \end{array}$$

Notice that the $G_{\phi ,\mathbf{x}}^{\left(T\right)}(\mathbf{x},\mathbf{y})$ denotes the truncated part and $G_{\phi ,\mathbf{x}}^{\left(R\right)}(\mathbf{x},\mathbf{y})$ refers to the reminder part. It is possible to write the U-statistic in the following way

$$\begin{array}{cc}\hfill u_n(\phi ,\mathbf{x},m_n)& =u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(T\right)}\right)+u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)\hfill \\ \hfill & =:u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)+u_n^{\left(R\right)}(\phi ,\mathbf{x},m_n).\hfill \end{array}$$

The first term of the right side $u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)$, as usual, is called the truncated part and the second one $u_n^{\left(R\right)}(\phi ,\mathbf{x},m_n)$ is the remainder part. Let us investigate the term $u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)$.

7.1. Truncated Part

In a similar way as in the preceding proof, we infer

$$\begin{array}{c}{\displaystyle \left|u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{(n-k)!}{n!}\left|\sum _{i\in I(k,n)}\left\{{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\prod _{j=1}^k{\delta }_{m_n}(x_j,X_{i_j})\right.\right.\hfill \\ \left.\left.\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}-\mathbb{E}\left[{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\prod _{j=1}^k{\delta }_{m_n}(x_j,X_{i_j})\right]\right\}\right|\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{(n-k)!}{n!}\left|\sum _{i\in I(k,n)}\left\{G_{\phi ,\mathbf{x}}^{\left(T\right)}({\mathbf{X}}_i,{\mathbf{Y}}_i)-\mathbb{E}\left[G_{\phi ,\mathbf{x}}^{\left(T\right)}({\mathbf{X}}_i,{\mathbf{Y}}_i)\right]\right\}\right|\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{(n-k)!}{n!}\left|\sum _{i\in I(k,n)}{H}^{\left(T\right)}({\mathbf{X}}_i,{\mathbf{Y}}_i)\right|,\hfill \end{array}$$

where

$$H^{\left(T\right)}(\mathbf{X},\mathbf{Y})=G_{\phi ,\mathbf{x}}^{\left(T\right)}(\mathbf{X},\mathbf{Y})-\mathbb{E}\left[G_{\phi ,\mathbf{x}}^{\left(T\right)}(\mathbf{X},\mathbf{Y})\right].$$

Similar to the proof of Theorem 1, we apply Lemma A1 on the function $H^{\left(T\right)}(·,·)$. Throughout the rest of the proof, we suppose that the function $G_{\phi ,\mathbf{x}}^{\left(T\right)}$ is symmetric. Moreover, it is clear that the function $H^{\left(T\right)}(·,·)$ is bounded by $2\lambda {\xi }_n^{1/q}{C}_1{s}_m$ by condition (C.2). We obviously remark that,

$$\theta =\mathbb{E}\left[H^{\left(T\right)}(\mathbf{X},\mathbf{Y})\right]=0$$

by design, and

$$\begin{array}{cc}\hfill {\sigma }^2& =Var\left(H^{\left(T\right)}(\mathbf{X},\mathbf{Y})\right)\le 2{\left(\lambda {\xi }_n^{1/q}{C}_1{s}_m\right)}^2.\hfill \end{array}$$

For any $\eta >0$ and m large enough, we obtain that

$$\begin{array}{cc}\hfill & \mathbb{P}\left(\left|u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hfill & \le 2exp\left[-\frac{n((s_{m}log\left(m\right))/m){\eta }^2}{4{\left(\lambda {\xi }_n^{1/q}{C}_1{s}_m\right)}^2+\frac{4}{3}\lambda {\xi }_n^{1/q}{C}_1{s}_m\eta \sqrt{(s_{m}log\left(m\right))/m}}\right].\hfill \end{array}$$

We can write

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n^{\left(T\right)}(\phi ,\mathbf{x},m_n)\right)\right|>2\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)-u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right.\right.\hfill \\ \left.\left.\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).\hfill \end{array}$$

(39)

Notice that

$$\begin{array}{cc}\hfill & \left|u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)-u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}\left|{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\left\{\prod _{j=1}^k{\delta }_{m_n}(x_j,X_{i_j})-\prod _{j=1}^k{\delta }_{m_n}(x_{\ell \left(x_j\right)},X_{i_j})\right\}\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}\left|{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\right|\left|{\delta }_{m_n}(\mathbf{x},{\mathbf{X}}_i)-{\delta }_{m_n}({\mathbf{x}}_{\ell \left(\mathbf{x}\right)},{\mathbf{X}}_i)\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}{C}_2{s}_m^{{\beta }_2}d{(\mathbf{x},{\mathbf{x}}_{\ell \left(\mathbf{x}\right)})}^{{\beta }_1}\left|{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}{C}_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\left|{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}\frac{1}{n}\sum _{j=1}^n{C}_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\left|{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\right|\hfill \\ \hfill & \le \frac{(n-k)!}{n!}\sum _{i\in I(k,n)}\frac{1}{n}\sum _{j=1}^n{W}_{j,T},\hfill \end{array}$$

where for $1\le j\le n$,

$$W_{j,T}:=C_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\left|{\phi }^{\left(T\right)}(Y_1,\dots ,Y_k)\right|,$$

and we can write

$$\begin{array}{cc}\hfill \mathbb{E}\left[W_{j,T}\right]& =C_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\mathbb{E}\left[\left|{\phi }^{\left(T\right)}(Y_1,\dots ,Y_k)\right|\right]\hfill \\ \hfill & =C_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\mathbb{E}\left[\mathbb{E}\left[{\phi }^{\left(T\right)}(Y_1,\dots ,Y_k)\mid \mathbf{X}=\mathbf{x}\right]\right],\hfill \end{array}$$

which means that for $2\le \nu \le q$:

$$\begin{array}{cc}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\mathbb{E}{\left[W_{j,T}\right]}^{\nu }\hfill & =\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\left(C_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\right)}^{\nu }\mathbb{E}{\left[\mathbb{E}\left[{\phi }^{\left(T\right)}(Y_{i_1},\dots ,Y_{i_k})\mid \mathbf{X}=\mathbf{x}\right]\right]}^{\nu }\hfill \end{array}$$

(40)

$$\begin{array}{cc}\le {\left(C_2{s}_m^{{\beta }_2}{ϵ}_n^{{\beta }_1}\right)}^{\nu }{\left(\lambda {\xi }_n\right)}^{\nu /q}{\mu }_q^{\nu /q}\hspace{0.17em}\hspace{0.17em}\hfill & \end{array}$$

(41)

$$\begin{array}{c}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le C_2^{\nu }{\left(\frac{s_{m}logm}{m}\right)}^{\nu /2}{\left(\lambda {\xi }_n\right)}^{\nu /q}{\mu }_q^{\nu /q}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le C_2^{\nu }{\lambda }^{\nu /q}{\left({\xi }_n\right)}^{\nu /q-\nu /2}{\left(s_m\right)}^{1-\nu /2}{\mu }_q^{\nu /q}{\left(s_m\right)}^{\nu -1}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \mathfrak{C}{s}_m^{(\nu -1)}.\hfill \end{array}$$

(42)

The passage from (40) to (41) is possible by the use of the Jensen’s inequality for the concave function $z^a$, for $0<a\le 1$, while (42) is by condition (C.5). Then, for $\nu \ge 2$

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\mathbb{E}{\left[W_{j,T}\right]}^{\nu }& \le & \mathfrak{C}{s}_m^{(\nu -1)},\hfill \end{array}$$

where $\mathfrak{C}>0$. Then, an application of classical inequality (see Corollary A.8-(ii) [48]) with $Z_i:=W_{j,T}$ and $a_n^2=s_m$, which gives us

$$u_m=a_n^{2}ln\left(m\right)m^{-1}=s_{m}ln\left(m\right)m^{-1},$$

and it is clear that $u_m\to 0$ with m by condition (C.2). Consequently, we obtain uniformly on $\mathbf{x}\in S_{\mathcal{X}}^k$:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)-u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

(43)

Now, we obtain from (43) that:

$$\begin{array}{c}\hfill {\displaystyle \left|\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)\right]\right|}\end{array}$$

$$\begin{array}{ccc}& \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=& \left|\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)\right]\right|\hfill \end{array}$$

(44)

$$\begin{array}{ccc}& \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le & \mathbb{E}\left|\left[u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)-u_n^{\left(T\right)}(\phi ,\mathbf{x};m_n)\right]\right|.\hfill \end{array}$$

(45)

Similar to the bounded case, the transition from (44) to (45) is due to Jensen’s inequality and some properties of the absolute value function. Furthermore, using the fact that for any constant a, $\mathbb{E}\left[a\right]=a$, we can directly conclude that

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

For sufficiently large n and large m, we infer that, for some $\eta >0$,

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x};m_n)-u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right.\right.}\hfill \\ \left.\left.\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\mathbb{E}\left[u_n(\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)};m_n)\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{x};m_n)\right]\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)=0.\hfill \end{array}$$

Continue, now, with (39), by imposing that the kernel function $G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}(·)$ is symmetric, the U-statistic is decomposed according to [8] decomposition, that is

$$\begin{array}{c}{\displaystyle u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell };m_n)-\mathbb{E}\left[u_n^{\left(T\right)}(\phi ,{\mathbf{x}}_{\ell };m_n)\right]}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\sum _{p=1}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell },m_n}^{\left(T\right)}\right)\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=k{u}_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)+\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right).\hfill \end{array}$$

(46)

Let us first start with the linear term. We have

$$k{u}_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)=\frac{k}{n}\sum _{j=1}^n{\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)({\mathbf{X}}_i,{\mathbf{Y}}_i).$$

From Hoeffding’s projection, we have

$$\begin{array}{cc}\hfill {\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)(x,y)& =\left\{\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\left((x,X_2,\dots ,X_k),(y,Y_2,\dots ,Y_k)\right)\right]-\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\left(\mathbf{X},\mathbf{Y}\right)\right]\right\}\hfill \\ \hfill & =\left\{\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\left(\mathbf{X},\mathbf{Y}\right)\right|(X_1,Y_1)=(x,y)]-\mathbb{E}\left[G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\left(\mathbf{X},\mathbf{Y}\right)\right]\right\}.\hfill \end{array}$$

Set

$$Z_i^{\left(T\right)}={\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)(X_i,Y_i).$$

We can clearly observe that $Z_i^{\left(T\right)}$ are independent and identically distributed random variables bounded by $2k\lambda {\xi }_n^{1/q}{C}_1{s}_m$ with zero mean and

$${\sigma }^2\le {\left(\lambda {\xi }_n^{1/q}{C}_1{s}_m\right)}^2.$$

An application of Bernstein’s inequality yields

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \sum _{i=1}^{d_n}\mathbb{P}\left(\underset{1\le {\ell }_i\le d_n}{max}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2d_{n}exp\left[-\frac{n((s_{m}log\left(m\right))/m){\eta }^2}{4{\left(\lambda {\xi }_n^{1/q}{C}_1{s}_m\right)}^2+\frac{4}{3}\lambda {\xi }_n^{1/q}{C}_1{s}_m\eta \sqrt{(s_{m}log\left(m\right))/m}}\right]\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le n^{\alpha -\tau {\eta }^2/C_4^{\prime }},\hfill \end{array}$$

for some positive constant $C_4^{\prime }$, resulting from the fact $m\le n$ and $log\left(m\right)\ge \tau log\left(n\right)$. This implies that

$$\begin{array}{cc}\hfill & \sum _{n\ge 1}\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hfill & \le \sum _{n\ge 1}{n}^{\alpha -\tau {\eta }^2/C_4^{\prime }}<\infty .\hfill \end{array}$$

Consequently, we obtain:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

Moving to the nonlinear term, we want to prove that for $2\le p\le k$:

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\left(\begin{array}{c}k\\ p\end{array}\right)\sqrt{m}\left|u_n^{\left(p\right)}\left({\pi }_{p,k}{\mathrm{G}}_{\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)}}^{\left(T\right)}\right)\right|}{\sqrt{s_{m}log\left(m\right)}}={\mathrm{O}}_{a.co}\left(1\right),$$

which implies that, for $1\le i\le k$ and $\ell =({\ell }_1,\dots ,{\ell }_k)$:

$$\underset{1\le {\ell }_i\le d_n}{max}\frac{\left(\begin{array}{c}k\\ p\end{array}\right)\sqrt{m}\left|u_n^{\left(p\right)}\left({\pi }_{p,k}{\mathrm{G}}_{\phi ,{\mathbf{x}}_{\ell \left(\mathbf{x}\right)}}^{\left(T\right)}\right)\right|}{\sqrt{s_{m}log\left(m\right)}}={\mathrm{O}}_{a.co}\left(1\right).$$

To prove the above-mentioned equation, we need to apply Proposition 1 of [125] (see Lemma A2). We can observe that $G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}$ is bounded by $C_1{s}_m\lambda {\xi }_n^{1/q}$, hence for $\eta >0$ we have

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(n^{1/2}\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\mathbb{P}\left(\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>n^{-1/2}\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\mathbb{P}\left(\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right),\hfill \end{array}$$

where ${\displaystyle {\epsilon }_0=\frac{\eta }{\sqrt{n}}}$. Now for ${\displaystyle t=\eta \sqrt{\frac{s_{m}log\left(m\right)}{m}}}$, Lemma A2 gives us:

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\left|\sum _{p=2}^k\frac{k!}{(k-p)!}{u}_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2exp\left(-\frac{t{(n-1)}^{1/2}}{2^{k+2}{k}^{k+1}\lambda {\xi }_n^{1/q}{C}_1{s}_m}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2exp\left(-\frac{\eta \sqrt{s_{m}log\left(m\right)/m}{(n-1)}^{1/2}}{2^{k+2}{k}^{k+1}\lambda {\xi }_n^{1/q}{C}_1{s}_m}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le 2exp\left(-\frac{\eta \sqrt{log\left(m\right)/m}{(n-1)}^{1/2}}{2^{k+2}{k}^{k+1}\lambda {\xi }_n^{1/q}{C}_1\sqrt{s_m}}\right).\hfill \end{array}$$

By the fact that $m\le n$ and $log\left(m\right)\ge \tau log\left(n\right)$, it follows that there exists $\eta >0$ in such a way that

$$\begin{array}{c}\hfill \mathbb{P}\left(\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\le n^{-\tau /2C_6},\end{array}$$

where

$$C_6=C^{″}{2}^{k+2}{k}^{k+1}\lambda {\xi }_n^{1/q}{C}_1\sqrt{s_m},$$

with $C^{″}>0.$ Therefore, for each ${\epsilon }_0>0$, $1\le i\le k$ and $\ell =({\ell }_1,\dots ,{\ell }_k)$:

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le d_n\underset{1\le {\ell }_i\le d_n}{max}\mathbb{P}\left(\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le n^{-k(\tau /2C_6)}.\hfill \end{array}$$

Consequently, we have

$$\begin{array}{c}{\displaystyle \sum _{n\ge 1}\mathbb{P}\left(\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\sum _{p=2}^k\left(\begin{array}{c}k\\ p\end{array}\right)u_n^{\left(p\right)}\left({\pi }_{p,k}\left(G_{\phi ,{\mathbf{x}}_{\ell }}^{\left(T\right)}\right)\right)\right|>{\epsilon }_0\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right)}\hfill \\ \le \sum _{n\ge 1}{n}^{\alpha -\tau /2C_6}\to 0\mathrm{a}\mathrm{s}n\to 0.\hfill \end{array}$$

7.2. Remainder Part

We now consider the remainder part, which is the U-process $u_n^{\left(R\right)}(\phi ,\mathbf{x},m_n)$ related on the unbounded kernel given by:

$$G_{\phi ,\mathbf{x}}^{\left(R\right)}(\mathbf{x},\mathbf{y})=G_{\phi ,\mathbf{x}}(\mathbf{x},\mathbf{y}){\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}$$

We have establish that the process is negligible, meaning that

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left|u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)-\mathbb{E}\left(u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)\right)\right|}{\sqrt{s_{m}log\left(m\right)}}=o_{a.co}\left(1\right).$$

(47)

Observe that for $\mathbf{x},\mathbf{y}\in {\mathcal{X}}^k,$

$$\begin{array}{cc}\hfill \left|G_{\phi ,\mathbf{x}}\right|& =\left|\phi \left(\mathbf{y}\right){\delta }_m(\mathbf{x},\mathbf{y})\right|\hfill \\ \hfill & \le C_1{s}_m\left|\phi \left(\mathbf{y}\right)\right|=:\tilde{F}\left(\mathbf{y}\right).\hfill \end{array}$$

Taking into account that $\tilde{F}$ is symmetric, we have:

$$\left|u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)\right|\le u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\tilde{F}>\lambda {\xi }_n^{1/q}\}}\right),$$

where $u_n^{\left(k\right)}\left(\tilde{F}\left(\mathbf{y}\right){\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\right)$ is a U-statistic based on the U-kernel $\tilde{F}{\mathbb{𝟙}}_{\{\phi >\lambda {\xi }_n^{1/q}\}}:$

$$\begin{array}{cc}\hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left|u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)\right|}{\sqrt{s_{m}log\left(m\right)}}& \le {\left(s_m^{-1}{\xi }_n\right)}^{1/2}{u}_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\tilde{F}>\lambda {\xi }_n^{1/q}\}}\right)\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill & \le C_7{\xi }_n{u}_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\tilde{F}>\lambda {\xi }_n^{1/q}\}}\right),\hfill \end{array}$$

(49)

and

$$\begin{array}{cc}\hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left|\mathbb{E}\left(u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)\right)\right|}{\sqrt{s_{m}log\left(m\right)}}& \le C_7{\xi }_n\mathbb{E}\left(u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\right)\hfill \\ \hfill & \le C_7\mathbb{E}\left({\tilde{F}}^{1+q}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right).\hfill \end{array}$$

Therefore, as $m⟶\infty$ when $n⟶\infty$, we have

$$\begin{array}{c}\hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left|\mathbb{E}\left(u_n^{\left(k\right)}\left(G_{\phi ,\mathbf{x}}^{\left(R\right)}\right)\right)\right|}{\sqrt{s_{m}log\left(m\right)}}=o\left(1\right).\end{array}$$

(50)

Hence to achieve the proof, it remains to establish that:

$$u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\right)=o_{a.co}\left({\left(s_m^{-1}{\xi }_n\right)}^{-1/2}\right).$$

(51)

An application of the Chebyshev’s inequality, for any $\eta >0$, gives

$$\begin{array}{c}{\displaystyle \mathbb{P}\left\{\left|u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)-\mathbb{E}\left(u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\right)\right|\ge \eta {\left(s_m^{-1}{\xi }_n\right)}^{-1/2}\right\}}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le {\eta }^{-2}\left(s_m^{-1}{\xi }_n\right)Var\left(u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\right)\le k{\eta }^{-2}{\xi }_n\mathbb{E}\left({\tilde{F}}^2{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{k}{n^2}{\eta }^{-2}{\left({\xi }_n\right)}^q\mathbb{E}\left({\tilde{F}}^2{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\le {\eta }^{\prime }\mathbb{E}\left({\tilde{F}}^3{\mathbb{𝟙}}_{\{\phi \left(\mathbf{Y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\frac{1}{n^2},\hfill \end{array}$$

so by using the fact that

$${\eta }^{\prime }\mathbb{E}\left({\tilde{F}}^3{\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\sum _{n\ge 1}\frac{1}{n^2}<\infty ,$$

we deduce that

$$\begin{array}{c}\hfill \sum _{n\ge 1}\mathbb{P}\left\{\left|u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\right)-\mathbb{E}\left(u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\right)\right|\ge \eta {\left(m{\xi }_n\right)}^{-1/2}\right\}<\infty .\end{array}$$

Finally, note that (48) implies

$$\mathbb{E}\left(u_n^{\left(k\right)}\left(\tilde{F}{\mathbb{𝟙}}_{\{\phi \left(\mathbf{y}\right)>\lambda {\xi }_n^{1/q}\}}\right)\right)=o\left({\left(s_m^{-1}{\xi }_n\right)}^{-1/2}\right).$$

The preceding results of the arbitrary choice of $\lambda >0$ gives that (51) holds, which, by combining with (50) and (48), completes the proof of (47). We finally obtain

$$\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(\phi ,\mathbf{x},m_n)-\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)\right|=O_{a.co}\left(\sqrt{\frac{s_{m}log\left(m\right)}{m}}\right).$$

Hence, the proof is complete. □

Proof of Theorem 3. 

The conclusion of Theorem 3 can be obtained from the results of Theorems 1 and 2. We have

$$\begin{array}{c}{\displaystyle \left|{\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)-\widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\left|\frac{u_n(\phi ,\mathbf{x};m_n)}{u_n(1,\mathbf{x};m_n)}-\frac{\mathbb{E}\left(u_n(\phi ,\mathbf{x};m_n)\right)}{\mathbb{E}\left(u_n(1,\mathbf{x};m_n)\right)}\right|\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{\left|u_n(\phi ,\mathbf{x};m_n)-\mathbb{E}\left(u_n(\phi ,\mathbf{x};m_n)\right)\right|}{\left|u_n(1,\mathbf{x};m_n)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\frac{\left|\mathbb{E}\left(u_n(\phi ,\mathbf{x};m_n)\right)\right|·\left|u_n(1,\mathbf{x};m_n)-\mathbb{E}\left(u_n(1,\mathbf{x};m_n)\right)\right|}{\left|u_n(1,\mathbf{x};m_n)\right|·\left|\mathbb{E}\left(u_n(1,\mathbf{x};m_n)\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=:{\mathcal{I}}_1+{\mathcal{I}}_2.\hfill \end{array}$$

Notice that, given the imposed hypothesis and previously obtained results, and for some $c_1,c_2$, we obtain:

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|u_n(1,\mathbf{x},m_n)\right|& =& c_1\mathrm{a}.\mathrm{c}\mathrm{o},\hfill \\ \hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\mathbb{E}\left(u_n(1,\mathbf{x},m_n)\right)\right|& =& c_2,\hfill \\ \hfill \underset{\mathbf{x}\in S_{\mathcal{X}}^m}{sup}\left|\mathbb{E}\left(u_n(\phi ,\mathbf{x},m_n)\right)\right|& =& O\left(1\right).\hfill \end{array}$$

Hence now, depending on whether the function $\phi (·)$ is bounded or unbounded, we can apply Theorem 1 or Theorem 2 (respectively) to handle both ${\mathcal{I}}_1$ and ${\mathcal{I}}_2$, and get for some $c^{″}>0$ with probability 1:

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left|\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)-\widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)\right|}{\sqrt{s_{m}log\left(m\right)}}}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left({\mathcal{I}}_1\right)}{\sqrt{s_{m}log\left(m\right)}}+\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\frac{\sqrt{m}\left({\mathcal{I}}_2\right)}{\sqrt{s_{m}log\left(m\right)}}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le c^{″}.\hfill \end{array}$$

Hence, the proof is complete. □

Proof of Theorem 4. 

Let $\gamma >0$ and $\mathbf{x}\in S_{\mathcal{X}}^k$. We have

$$\begin{array}{cc}\hfill \widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)-r^{\left(k\right)}\left(\phi ,\mathbf{x}\right)& =\frac{\mathbb{E}\left[u_n(\phi ,\mathbf{x},m_n)\right]}{\mathbb{E}\left[u_n(1,\mathbf{x},m_n)\right]}-r^{\left(k\right)}\left(\phi ,\mathbf{x}\right).\hfill \end{array}$$

Notice that

$$\begin{array}{ccc}\hfill \widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)& =& \frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\mathbb{E}\left[\phi (Y_1,\dots ,Y_k)\prod _{j=1}^k{\delta }_{m_n}(x_j,X_j)\right]\hfill \\ & =& {\displaystyle \frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}{\int }_{{\mathcal{X}}^k}{r}^{\left(k\right)}(\phi ,\mathbf{t}){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right),}\hfill \end{array}$$

where for $\mathbf{t}=(t_1,\dots ,t_k)$ we denote $\mu \left(\mathbf{dt}\right):=\mu \left(d{t}_1\right)\cdots \mu \left(d{t}_k\right),$ and

$${\displaystyle \tilde{f}\left(\mathbf{t}\right):=\prod _{j=1}^{k}f\left(t_j\right).}$$

Which means that

$$\begin{array}{c}{\displaystyle \widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)-r^{\left(k\right)}\left(\phi ,\mathbf{x}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\left({\int }_{{\mathcal{X}}^k}{r}^{\left(k\right)}(\phi ,\mathbf{t}){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)-r^{\left(k\right)}(\phi ,\mathbf{x}){\int }_{{\mathcal{X}}^k}{\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\left({\int }_{{\mathcal{X}}^k}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}:=I_1\left(\mathbf{x}\right)+I_2\left(\mathbf{x}\right),\hfill \end{array}$$

where

$$I_1\left(\mathbf{x}\right):=\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{dt}\right),$$

(52)

and

$$I_2\left(\mathbf{x}\right):=\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}{\int }_{\overline{\mathbb{B}}(\mathbf{x},\gamma )}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{dt}\right).$$

(53)

Therefore, we need to study the asymptotic behavior of both ${\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left(I_1\left(\mathbf{x}\right)\right)}$ and ${\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left(I_2\left(\mathbf{x}\right)\right)}$ to obtain the desired result.

Let us start with the term ${\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left(I_1\left(\mathbf{x}\right)\right)}$, we have

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_1\left(\mathbf{x}\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)\right|\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}\left|\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\right|\mu \left(\mathbf{d}\mathbf{t}\right),\hfill \end{array}$$

taking into account the fact that the density function $f(·)$ is bounded, and by condition (C.9), we get:

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_1\left(\mathbf{x}\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}{C}_{3}d(\mathbf{x},\mathbf{t}){\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f{C}_3\gamma }{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}{\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right),\hfill \end{array}$$

(54)

hence, the term in (54) can be chosen smaller than $2\epsilon$ as $m\to \infty$ by using the condition (8).

To investigate the term ${\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left(I_2\left(\mathbf{x}\right)\right)}$, notice that

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_2\left(\mathbf{x}\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}{\int }_{\overline{\mathbb{B}}(\mathbf{x},\gamma )}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)\right|\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f{C}_3}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\overline{\mathbb{B}}(\mathbf{x},\gamma )}d(\mathbf{x},\mathbf{t}){\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right).\hfill \end{array}$$

(55)

By condition (11), we conclude that

$${\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_2\left(\mathbf{x}\right)\right|\to 0\mathrm{a}\mathrm{s}m\to \infty .}$$

This concludes the proof of the Theorem. □

Proof of Theorem 5. 

Following the same steps as the proof of Theorem 4, we can write directly:

$$\begin{array}{c}{\displaystyle \widehat{\mathbb{E}}\left({\widehat{r}}_n^{\left(k\right)}(\phi ,\mathbf{x};m_n)\right)-r^{\left(k\right)}\left(\phi ,\mathbf{x}\right)}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\left({\int }_{{\mathcal{X}}^k}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)\right).\hfill \end{array}$$

Taking into account conditions (C.8), we can easily deduce that

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|\frac{1}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\left({\int }_{{\mathcal{X}}^k}\left(r^{\left(k\right)}(\phi ,\mathbf{t})-r^{\left(k\right)}(\phi ,\mathbf{x})\right){\delta }_m(\mathbf{x},\mathbf{t})\tilde{f}\left(\mathbf{t}\right)\mu \left(\mathbf{d}\mathbf{t}\right)\right)\right|}\hfill \\ \le \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_1\left(\mathbf{x}\right)\right|+\left|I_2\left(\mathbf{x}\right)\right|,\hfill \end{array}$$

where $I_1\left(\mathbf{x}\right)$ and $I_2\left(\mathbf{x}\right)$ are defined in (52) and (53), respectively. Presently, Equation (54) gives us

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_1\left(\mathbf{x}\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}{C}_{3}d(\mathbf{x},\mathbf{t}){\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f{C}_3{D}_m}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\mathbb{B}(\mathbf{x},\gamma )}{\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le O\left(D_m\right),\hfill \end{array}$$

by conditions (C.8) and (C.1). On the other hand, Equation (55) gives us:

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}\left|I_2\left(\mathbf{x}\right)\right|}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f{C}_3}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\overline{\mathbb{B}}(\mathbf{x},\gamma )}d(\mathbf{x},\mathbf{t}){\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \frac{C_f{C}_3{D}_m}{\mathbb{E}\left[{\delta }_m(\mathbf{x},\mathbf{X})\right]}\underset{\mathbf{x}\in S_{\mathcal{X}}^k}{sup}{\int }_{\overline{\mathbb{B}}(\mathbf{x},\gamma )}{\delta }_m(\mathbf{x},\mathbf{t})\mu \left(\mathbf{d}\mathbf{t}\right)\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le O\left(D_m\right),\hfill \end{array}$$

by condition (C.8). This completes the proof of the theorem. □