Skip to main content
SpringerLink
Log in

Nonparametric Recursive Estimation for Multivariate Derivative Functions by Stochastic Approximation Method

  • Published:
Sankhya A Aims and scope Submit manuscript

Abstract

Important information concerning a multivariate data set, such as modal regions, is contained in the derivatives of the probability density or regression functions. Despite this importance, nonparametric estimation of higher order derivatives of the density or regression functions have received only relatively scant attention. The main purpose of the present work is to investigate general recursive kernel type estimators of function derivatives. We establish the central limit theorem for the proposed estimators. We discuss the optimal choice of the bandwidth by using the plug in methods. We obtain also the pointwise MDP of these estimators. Finally, we investigate the performance of the methodology for small samples through a short simulation study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

References

  • Allaoui, S., Bouzebda, S., Chesneau, C. and Liu, J. (2021). Uniform almost sure convergence and asymptotic distribution of the wavelet-based estimators of partial derivatives of multivariate density function under weak dependence. J. Nonparametr. Stat. 33, 170–196.

    Article  MATH  Google Scholar 

  • Altman, N. and Leger, C. (1995). Bandwidth selection for kernel distribution function estimation. J. Statist. Plann. Inference 46, 195–214.

    Article  MATH  Google Scholar 

  • Bojanic, R. and Seneta, E. (1973). A unified theory of regularly varying sequences. Math. Z. 134, 91–106.

    Article  MATH  Google Scholar 

  • Bouzebda, S. (2012). On the strong approximation of bootstrapped empirical copula processes with applications. Math. Methods Statist. 21, 153–188.

    Article  MATH  Google Scholar 

  • Bouzebda, S. and Didi, S. (2020). Some results about kernel estimators for function derivatives based on stationary and ergodic continuous time processes with applications. Communications in Statistics - Theory and Methods, 1–48.

  • Bouzebda, S. and El-hadjali, T. (2020). Uniform convergence rate of the kernel regression estimator adaptive to intrinsic dimension in presence of censored data. J. Nonparametr. Stat. 32, 864–914.

    Article  MATH  Google Scholar 

  • Bouzebda, S. and Elhattab, I. (2011). Uniform-in-bandwidth consistency for kernel-type estimators of Shannon’s entropy. Electron. J. Stat. 5, 440–459.

    Article  MATH  Google Scholar 

  • Bouzebda, S., Elhattab, I. and Seck, C. T. (2018). Uniform in bandwidth consistency of nonparametric regression based on copula representation. Statist. Probab. Lett. 137, 173–182.

    Article  MATH  Google Scholar 

  • Bouzebda, S., Elhattab, I. and Nemouchi, B. (2021). On the uniform-in-bandwidth consistency of the general conditional U-statistics based on the copula representation. J. Nonparametr. Stat. 33, 321–358.

    Article  MATH  Google Scholar 

  • Bouzebda, S. and Nemouchi, B. (2020). Uniform consistency and uniform in bandwidth consistency for nonparametric regression estimates and conditional U-statistics involving functional data. J. Nonparametr. Stat. 32, 452–509.

    Article  MATH  Google Scholar 

  • Charnigo, R., Hall, B. and Srinivasan, C. (2011). A generalized cp criterion for derivative estimation. Technometrics 53, 238–253.

    Article  Google Scholar 

  • Deheuvels, P. (1973). Sur l’estimation séquentielle de la densité. C.R. Acad. Sci. Paris Ser. A-B 276, 1119–1121.

    MATH  Google Scholar 

  • Deheuvels, P. (2011). One bootstrap suffices to generate sharp uniform bounds in functional estimation. Kybernetika (Prague) 47, 855–865.

    MATH  Google Scholar 

  • Delaigle, A. and Gijbels, I. (2004). Practical bandwidth selection in deconvolution kernel density estimation. Comput. Statist. Data Anal. 45, 249–267.

    Article  MATH  Google Scholar 

  • Devroye, L. (1987). A course in density estimation, volume 14 of Progr. Probab. Statist. Birkhäuser Boston Inc., Boston.

    Google Scholar 

  • Devroye, L. and Györfi, L. (1985). Nonparametric density estimation. Wiley Series in Probability and Mathematical Statistics: Tracts on Probability and Statistics. Wiley, New York. The L1 view.

    Google Scholar 

  • Devroye, L. and Lugosi, G. (2001). Combinatorial methods in density estimation Springer Series in Statistics. Springer, New York.

    MATH  Google Scholar 

  • Dobrovidov, A. V. and Rud’ ko, I. M. (2010). Choice of bandwidth in the kernel function in the nonparametric estimation of the derivative of density by the smoothed cross-validation method. Avtomat. i Telemekh., (2), 42–58.

  • Duflo, M. (1997). Random iterative models. Collection Applications of Mathematics. Springer, Berlin.

    Google Scholar 

  • Eggermont, P. P. B. and LaRiccia, V. N. (2001). Maximum penalized likelihood estimation. Vol. I Springer Series in Statistics. Springer, New York. Density estimation.

    MATH  Google Scholar 

  • Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab.13, 1–37.

    Article  MATH  Google Scholar 

  • Einmahl, U. and Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Ann. Statist. 94, 236–238.

    MATH  Google Scholar 

  • Eubank, R. L. and Speckman, P. L. (1993). Confidence bands in nonparametric regression. J. Amer. Statist. Assoc. 88, 1287–1301.

    Article  MATH  Google Scholar 

  • Galambos, J. and Seneta, E. (1973). Regularly varying sequences. J. Amer. Math. Soc. 41, 110–116.

    MATH  Google Scholar 

  • Genovese, C. R., Perone-Pacifico, M., Verdinelli, I. and Wasserman, L. A. (2013). Nonparametric inference for density modes. CoRR, arXiv:1312.7567.

  • Härdle, W. (1990). Applied nonparametric regression volume 19 of Econometric Society Monographs. Cambridge University Press, Cambridge.

    Google Scholar 

  • Mokkadem, A., Pelletier, M. and Thiam, B. (2006). Large and moderate deviations principles for recursive kernel estimator of a multivariate density and its partial derivatives. Serdica Math. J. 32, 323–354.

    MATH  Google Scholar 

  • Mokkadem, A. and Pelletier, M. (2007). A companion for the Kiefer-Wolfowitz-Blum stochastic approximation algorithm. Ann. Statist. 35, 1749–1772.

    Article  MATH  Google Scholar 

  • Mokkadem, A., Pelletier, M. and Slaoui, Y (2009a). The stochastic approximation method for the estimation of a multivariate probability density. J. Statist. Plann. Inference 139, 2459–2478.

    Article  MATH  Google Scholar 

  • Mokkadem, A., Pelletier, M. and Slaoui, Y (2009b). Revisiting Révész’s stochastic approximation method for the estimation of a regression function. ALEA Lat. Am. J. Probab. Math. Stat. 6, 63–114.

    MATH  Google Scholar 

  • Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and its Applications 10, 186–190.

    Article  MATH  Google Scholar 

  • Nadaraya, È. A. (1989). Nonparametric estimation of probability densities and regression curves volume 20 of Mathematics and its Applications (Soviet Series). Kluwer Academic Publishers Group, Dordrecht. Translated from the Russian by Samuel Kotz.

    Google Scholar 

  • Noh, Y. -K., Sugiyama, M., Liu, S., Plessis, M. C. d., Park, F. C. and Lee, D. D. (2018). Bias reduction and metric learning for nearest-neighbor estimation of kullback-leibler divergence. Neural Computation, 30(7), 1930–1960. PMID 29902113.

  • Park, C. and Kang, K. -H. (2008). SiZer analysis for the comparison of regression curves. Comput. Statist. Data Anal. 52, 3954–3970.

    Article  MATH  Google Scholar 

  • Puhalskii, A. (1994). The method of stochastic exponentials for large deviations. Stochastic Process. Appl. 54, 45–70.

    Article  Google Scholar 

  • Ramsay, J. O. and Silverman, B. W. (2002). Applied functional data analysis. Springer Series in Statistics. Springer, New York. Methods and case studies.

    Google Scholar 

  • Ramsay, J. O. and Silverman, B. W. (2005). Functional data analysis, 2nd edition. Springer Series in Statistics. Springer, New York.

    Book  MATH  Google Scholar 

  • Révész, P. (1973). Robbins-monro procedure in a Hilbert space and its application in the theory of learning processes. I. Studia Sci. Math. Hungar. 8, 391–398.

    MATH  Google Scholar 

  • Révész, P. (1977). How to apply the method of stochastic approximation in the non-parametric estimation of a regression function. Math. Operationsforsch. Statist. Ser. Statist 8, 119–126.

    MATH  Google Scholar 

  • Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90, 1257–1270.

    Article  MATH  Google Scholar 

  • Scott, D. W. (1992). Multivariate density estimation. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Wiley, New York. Theory, practice, and visualization: A Wiley-Interscience Publication.

    Book  Google Scholar 

  • Silverman, B. W. (1986). Density estimation for statistics and data analysis. Chapman and Hall, London.

    MATH  Google Scholar 

  • Singh, R. S. (1977). Applications of estimators of a density and its derivatives to certain statistical problems. J. Roy. Statist. Soc. Ser. B 39, 357–363.

    MATH  Google Scholar 

  • Slaoui, Y. (2013). Large and moderate principles for recursive kernel density estimators defined by stochastic approximation method. Serdica Math. J. 39, 53–82.

    MATH  Google Scholar 

  • Slaoui, Y. (2014a). Bandwidth selection for recursive kernel density estimators defined by stochastic approximation method. J. Probab. Stat., 2014, ID 739640. https://doi.org/10.1155/2014/739640.

  • Slaoui, Y. (2014b). The stochastic approximation method for the estimation of a distribution function. Math. Methods Statist. 23, 306–325.

    Article  MATH  Google Scholar 

  • Slaoui, Y. (2015a). Large and moderate deviation principles for averaged stochastic approximation method for the estimation of a regression function. Serdica Math. J. 41, 307–328.

    MATH  Google Scholar 

  • Slaoui, Y. (2015b). Plug-in Bandwidth selector for recursive kernel regression estimators defined by stochastic approximation method. Stat. Neerl. 69, 483–509.

    Article  Google Scholar 

  • Slaoui, Y. (2019). Wild bootstrap bandwidth selection of recursive nonparametric relative regression for independent functional data. J. Multivariate Anal.173, 494–511.

    Article  MATH  Google Scholar 

  • Slaoui, Y. (2020). Recursive nonparametric regression estimation for independent functional data. Statist. Sinica. 30. https://doi.org/10.5705/ss.202018.0069.

  • Tapia, R. A. and Thompson, J. R. (1978). Nonparametric probability density estimation, volume 1 of Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore.

    Google Scholar 

  • Tsybakov, A. B. (1990). Recurrent estimation of the mode of a multidimensional distribution. Probl. Inf. Transm. 8, 119–126.

    Google Scholar 

  • Vasil’ev, V. A., Dobrovidov, A. V. and Koshkin, G.M.s (2004). Neparametricheskoe otsenivanie funktsionalov ot raspredeleniĭ statsionarnykh posledovatel’ nosteĭ. “Nauka”, Moscow.

    MATH  Google Scholar 

  • Wand, M. P. and Jones, M. C. (1995). Kernel smoothing volume 60 of Monographs on Statistics and Applied Probability. Chapman and Hall Ltd., London.

    Book  Google Scholar 

  • Watson, G. S. (1964). Smooth regression analysis. Sankhya A 26, 359–372.

    MATH  Google Scholar 

  • Wegman, E. J. and Davies, H. I. (1979). Remarks on some recursive estimators of a probability density. Ann. Statist. 7, 316–327.

    Article  MATH  Google Scholar 

  • Wertz, W. (1978). Statistical density estimation: a survey volume 13 of Angewandte Statistik und Ökonometrie [Applied Statistics and Econometrics]. Göttingen, Vandenhoeck & Ruprecht. With German and French summaries.

    Google Scholar 

  • Wolverton, C. and Wagner, T. (1969). Asymptotically optimal discriminant functions for pattern classification. IEEE Trans. Inform. Theory 15, 258–265.

    Article  MATH  Google Scholar 

  • Yamato, H. (1971). Sequential estimation of a continuous probability density function and mode. IBull Math. Statist. 14, 1–12.

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors are indebted to the Editor-in-Chief, Associate Editor and the referee for their very valuable comments, suggestions careful reading of the article which led to a considerable improvement of the manuscript. The authors thank Mr. Issam Elhattab for his important help with the simulation section.

Funding

This work benefited from the financial support of the GDR 3477 GeoSto.

Author information

Authors and Affiliations

  1. LMAC, Université de Technologie de Compiègne, Compiègne, France

    Salim Bouzebda

  2. Laboratoire de Mathématiques et Applications, University de Poitiers, Futuroscope, Chasseneuil, France

    Yousri Slaoui

Authors
  1. Salim Bouzebda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yousri Slaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salim Bouzebda.

Additional information

Disclosure statement

No potential conflict of interest was reported by the authors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouzebda, S., Slaoui, Y. Nonparametric Recursive Estimation for Multivariate Derivative Functions by Stochastic Approximation Method. Sankhya A 85, 658–690 (2023). https://doi.org/10.1007/s13171-021-00272-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-021-00272-1

Keywords

PACS Nos

Search

Navigation