Penalization versus Goldenshluger − Lepski strategies in warped bases regression

Gaëlle Chagny

ESAIM: Probability and Statistics (2013)

  • Volume: 17, page 328-358
  • ISSN: 1292-8100

Abstract

top
This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, f ^ l = h ^ l ∘ G f̂l = ĥl°G if Gis known, or f ^ l = h ^ l ∘ G ^ f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.

How to cite

top

Chagny, Gaëlle. "Penalization versus Goldenshluger − Lepski strategies in warped bases regression." ESAIM: Probability and Statistics 17 (2013): 328-358. <http://eudml.org/doc/273619>.

@article{Chagny2013,
abstract = {This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, $\hat\{f\}_l=\hat\{h\}_l\circ G$f̂l = ĥl°G if Gis known, or $\hat\{f\}_l=\hat\{h\}_l\circ \hat\{G\}$f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.},
author = {Chagny, Gaëlle},
journal = {ESAIM: Probability and Statistics},
keywords = {adaptive estimator; model selection; nonparametric regression estimation; warped bases},
language = {eng},
pages = {328-358},
publisher = {EDP-Sciences},
title = {Penalization versus Goldenshluger − Lepski strategies in warped bases regression},
url = {http://eudml.org/doc/273619},
volume = {17},
year = {2013},
}

TY - JOUR
AU - Chagny, Gaëlle
TI - Penalization versus Goldenshluger − Lepski strategies in warped bases regression
JO - ESAIM: Probability and Statistics
PY - 2013
PB - EDP-Sciences
VL - 17
SP - 328
EP - 358
AB - This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, $\hat{f}_l=\hat{h}_l\circ G$f̂l = ĥl°G if Gis known, or $\hat{f}_l=\hat{h}_l\circ \hat{G}$f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.
LA - eng
KW - adaptive estimator; model selection; nonparametric regression estimation; warped bases
UR - http://eudml.org/doc/273619
ER -

References

top
  1. [1] A. Antoniadis, G. Grégoire and P. Vial, Random design wavelet curve smoothing. Statist. Probab. Lett.35 (1997) 225–232. Zbl0889.62029MR1484959
  2. [2] J.Y. Audibert and O. Catoni, Robust linear least squares regression. Ann. Stat. (2011) (to appear), arXiv:1010.0074. Zbl1231.62126MR2906886
  3. [3] J.Y. Audibert and O. Catoni, Robust linear regression through PAC-Bayesian truncation. Preprint, arXiv:1010.0072. 
  4. [4] Y. Baraud, Model selection for regression on a random design. ESAIM: PS 6 (2002) 127–146. Zbl1059.62038MR1918295
  5. [5] A. Barron, L. Birgé and P. Massart, Risk bounds for model selection via penalization. Probab. Theory Relat. Fields113 (1999) 301–413. Zbl0946.62036MR1679028
  6. [6] J.P. Baudry, C. Maugis and B. Michel, Slope heuristics: overview and implementation. Stat. Comput. 22-2 (2011) 455–470. Zbl1322.62007MR2865029
  7. [7] L. Birgé, Model selection for Gaussian regression with random design. Bernoulli10 (2004) 1039–1051. Zbl1064.62030MR2108042
  8. [8] L. Birgé and P. Massart, Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli4 (1998) 329–375. Zbl0954.62033MR1653272
  9. [9] L. Birgé and P. Massart, Minimal penalties for gaussian model selection. Probab. Theory Relat. Fields138 (2006) 33–73. Zbl1112.62082MR2288064
  10. [10] E. Brunel and F. Comte, Penalized contrast estimation of density and hazard rate with censored data. Sankhya67 (2005) 441–475. Zbl1192.62102MR2235573
  11. [11] E. Brunel, F. Comte and A. Guilloux, Nonparametric density estimation in presence of bias and censoring. Test18 (2009) 166–194. Zbl1203.62052MR2495970
  12. [12] T.T. Cai and L.D. Brown, Wavelet shrinkage for nonequispaced samples. Ann. Stat.26 (1998) 1783–1799. Zbl0929.62047MR1673278
  13. [13] G. Chagny, Régression: bases déformées et sélection de modèles par pénalisation et méthode de Lepski. Preprint, hal-00519556 v2. 
  14. [14] F. Comte and Y. Rozenholc, A new algorithm for fixed design regression and denoising. Ann. Inst. Stat. Math.56 (2004) 449–473. Zbl1057.62030MR2095013
  15. [15] R.A. DeVore and G. Lorentz, Constructive approximation, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 303. Springer-Verlag, Berlin (1993). Zbl0797.41016MR1261635
  16. [16] D.L. Donoho, I.M. Johnstone, G. Kerkyacharian and D. Picard, Wavelet shrinkage: asymptopia? With discussion and a reply by the authors. J. Roy. Stat. Soc., Ser. B 57 (1995) 301–369. Zbl0827.62035MR1323344
  17. [17] A. Dvoretzky, J. Kiefer and J. Wolfowitz, Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann. Math. Stat.27 (1956) 642–669. Zbl0073.14603MR83864
  18. [18] S. Efromovich, Nonparametric curve estimation: Methods, theory, and applications. Springer Series in Statistics, Springer-Verlag, New York (1999) xiv+411 Zbl0935.62039MR1705298
  19. [19] J. Fan and I. Gijbels, Variable bandwidth and local linear regression smoothers. Ann. Stat.20 (1992) 2008–2036. Zbl0765.62040MR1193323
  20. [20] S. Gaïffas, On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM: PS 11 (2007) 344–364. Zbl1187.62074MR2339297
  21. [21] A. Goldenshluger and O. Lepski, Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality. Ann. Stat.39 (2011) 1608–1632. Zbl1234.62035MR2850214
  22. [22] G.K. Golubev and M. Nussbaum, Adaptive spline estimates in a nonparametric regression model. Teor. Veroyatnost. i Primenen. ( Russian) 37 (1992) 554–561; translation in Theor. Probab. Appl. 37 (1992) 521–529. Zbl0787.62044MR1214361
  23. [23] W. Härdle and A. Tsybakov, Local polynomial estimators of the volatility function in nonparametric autoregression. J. Econ.81 (1997) 223–242. Zbl0904.62047MR1484586
  24. [24] G. Kerkyacharian and D. Picard, Regression in random design and warped wavelets. Bernoulli10 (2004) 1053–1105. Zbl1067.62039MR2108043
  25. [25] T. Klein and E. Rio, Concentration around the mean for maxima of empirical processes. Ann. Probab.33 (2005) 1060–1077. Zbl1066.60023MR2135312
  26. [26] M. Köhler and A. Krzyzak, Nonparametric regression estimation using penalized least squares. IEEE Trans. Inf. Theory47 (2001) 3054–3058. Zbl1008.62580MR1872867
  27. [27] C. Lacour, Adaptive estimation of the transition density of a particular hidden Markov chain. J. Multivar. Anal.99 (2008) 787–814. Zbl1286.62071MR2405092
  28. [28] E. Nadaraya, On estimating regression. Theory Probab. Appl.9 (1964) 141–142. Zbl0136.40902
  29. [29] T.-M. Pham Ngoc, Regression in random design and Bayesian warped wavelets estimators. Electron. J. Stat.3 (2009) 1084–1112. Zbl1326.62077MR2566182
  30. [30] A.B. Tsybakov, Introduction à l’estimation non-paramétrique, Mathématiques & Applications (Berlin), vol. 41. Springer-Verlag, Berlin (2004). Zbl1029.62034MR2013911
  31. [31] G.S. Watson, Smooth regression analysis. Sankhya A26 (1964) 359–372. Zbl0137.13002MR185765
  32. [32] M. Wegkamp, Model selection in nonparametric regression. Ann. Stat.31 (2003) 252–273. Zbl1019.62037MR1962506

NotesEmbed ?

top

You must be logged in to post comments.