Fixed design local polynomial smoothing and bandwidth selection for right censored data

Abstract

The local polynomial smoothing of the Kaplan–Meier estimate for fixed designs is explored and analyzed. The first benefit, in comparison to classical convolution kernel smoothing, is the development of boundary aware estimates of the distribution function, its derivatives and integrated derivative products of any arbitrary order. The advancements proceed by developing asymptotic mean integrated square error optimal solve-the-equation plug-in bandwidth selectors for the estimates of the distribution function and its derivatives, and as a byproduct, a mean square error optimal bandwidth rule for integrated derivative products. The asymptotic properties of all methodological contributions are quantified analytically and discussed in detail. Three real data analyses illustrate the benefits of the proposed methodology in practice. Finally, numerical evidence is provided on the finite sample performance of the proposed technique with reference to benchmark estimates.

Introduction

Let $T$ denote a continuous lifetime variable with cumulative distribution function (c.d.f.) $F_T\left(t\right)=P(T\le t)$. Frequently the available data are beyond the experimenter’s control and come in the form of scatterplot observations. For example, this is the case in lifetable analyses in the actuarial science, in data analyses in demography e.t.c., see Müller et al. (1997) and Wang et al. (1998). In such an occasion, the coordinates of the available data pairs consist of the response, which is usually an empirical estimate of the target curve, and the center of the associated time interval at which the curve is being estimated. Still, continuous estimates are desirable, especially when the analysis additionally depends on the estimate’s derivatives. For this reason, the present research considers the local polynomial smoothing of the well-known Kaplan–Meier estimate (Kaplan and Meier, 1958), with first objective to provide continuous, boundary aware estimates for the distribution function, its derivatives of any arbitrary order and integrated c.d.f. derivative products for fixed designs under the random right censorship model. The reasoning for pursuing this approach becomes immediately obvious when observing that smoothing of scatterplot data intrinsically corresponds to formulating a reasonable nonparametric regression problem. The asymptotic unbiasedness property of the Kaplan–Meier estimate together with its strong representation as the underlying c.d.f. plus an asymptotically negligible error term, prompts its use as the response. At the same time, the center of each equidistant time interval in which the observed data range is split is used as the design. Hence, application of the local polynomial technique yields estimates of $F_T^{\left(\nu \right)},\nu =0,1,\dots$ by matching the coefficients of polynomials fitted locally — through kernel weighted least squares — and the derivatives of $F_T$ in a Taylor expansion of the regression function at a nearby point; precise formulation and details are provided in Section 2. This approach enables the development, also in Section 2, of local polynomial estimates for integrated c.d.f. derivative products of any arbitrary order. These are useful on their own right since they are necessary for the implementation of automatic bandwidth selectors, in estimation of population characteristics, statistical distance measures and in a variety of other settings.

Multiple benefits arise from the local polynomial smoothing of the Kaplan–Meier estimate. First, its definition does not involve a bandwidth and thus its use as the response in the aforementioned nonparametric regression problem greatly simplifies implementation of the resulting estimates which now depend on just one bandwidth; this is in contrast to the traditional approach which needs two bandwidths. In terms of performance, the Asymptotic Mean Integrated Square Error (AMISE) and central limit theorem for the estimates of $F_T^{\left(\nu \right)}$, quantified analytically in Section 3, are valid throughout the region of estimation and imply the absence of inflated bias at the endpoints. Further, the asymptotic properties of the integrated derivative product estimates, also quantified in Section 3, ensure efficient estimation of the functionals as opposed to using conventional kernel smoothers. A subsequent advantage thus results by their utilization in developing (in Section 4) a solve-the-equation AMISE-optimal plug-in bandwidth rule applicable to all estimates proposed here. The rule is built as a direct extension of the corresponding density estimation bandwidth selector for complete data proposed in Cheng (1997). The gain is its stable performance across the region of estimation; this is also reflected in its asymptotic properties, quantified analytically together with its convergence rate and asymptotic distribution in Section 4. It is worth noting here that the literature is rather thin on AMISE optimal bandwidth rules for convolution smoother estimates for right censored data. Since the plug-in rule proposed here is also applicable to classical kernel approach, it can also be thought as filling this important gap in the literature.

Section 5 investigates the finite sample performance of the proposed methodology. First, the analysis of three real world data sets illustrates how the proposed technique can help in capturing data patterns that remain undiscoverable either by the conventional kernel smoothing approach or by parametric estimates. Finally, distributional data are used to simulate and compare the finite sample MISE performance of the proposed estimates in comparison to frequently used estimates in the literature and in practice.