Penalization versus Goldenshluger − Lepski strategies in warped bases regression

Gaëlle Chagny

ESAIM: Probability and Statistics (2013)

  • Volume: 17, page 328-358
  • ISSN: 1292-8100

Abstract

top
This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, f ^ l = h ^ l ∘ G f̂l = ĥl°G if Gis known, or f ^ l = h ^ l ∘ G ^ f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.

How to cite

top

Chagny, Gaëlle. "Penalization versus Goldenshluger − Lepski strategies in warped bases regression." ESAIM: Probability and Statistics 17 (2013): 328-358. <http://eudml.org/doc/273619>.

@article{Chagny2013,
abstract = {This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, $\hat\{f\}_l=\hat\{h\}_l\circ G$f̂l = ĥl°G if Gis known, or $\hat\{f\}_l=\hat\{h\}_l\circ \hat\{G\}$f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.},
author = {Chagny, Gaëlle},
journal = {ESAIM: Probability and Statistics},
keywords = {adaptive estimator; model selection; nonparametric regression estimation; warped bases},
language = {eng},
pages = {328-358},
publisher = {EDP-Sciences},
title = {Penalization versus Goldenshluger − Lepski strategies in warped bases regression},
url = {http://eudml.org/doc/273619},
volume = {17},
year = {2013},
}

TY - JOUR
AU - Chagny, Gaëlle
TI - Penalization versus Goldenshluger − Lepski strategies in warped bases regression
JO - ESAIM: Probability and Statistics
PY - 2013
PB - EDP-Sciences
VL - 17
SP - 328
EP - 358
AB - This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, $\hat{f}_l=\hat{h}_l\circ G$f̂l = ĥl°G if Gis known, or $\hat{f}_l=\hat{h}_l\circ \hat{G}$f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.
LA - eng
KW - adaptive estimator; model selection; nonparametric regression estimation; warped bases
UR - http://eudml.org/doc/273619
ER -

References

top
  1. [1] A. Antoniadis, G. Grégoire and P. Vial, Random design wavelet curve smoothing. Statist. Probab. Lett.35 (1997) 225–232. Zbl0889.62029MR1484959
  2. [2] J.Y. Audibert and O. Catoni, Robust linear least squares regression. Ann. Stat. (2011) (to appear), arXiv:1010.0074. Zbl1231.62126MR2906886
  3. [3] J.Y. Audibert and O. Catoni, Robust linear regression through PAC-Bayesian truncation. Preprint, arXiv:1010.0072. 
  4. [4] Y. Baraud, Model selection for regression on a random design. ESAIM: PS 6 (2002) 127–146. Zbl1059.62038MR1918295
  5. [5] A. Barron, L. Birgé and P. Massart, Risk bounds for model selection via penalization. Probab. Theory Relat. Fields113 (1999) 301–413. Zbl0946.62036MR1679028
  6. [6] J.P. Baudry, C. Maugis and B. Michel, Slope heuristics: overview and implementation. Stat. Comput. 22-2 (2011) 455–470. Zbl1322.62007MR2865029
  7. [7] L. Birgé, Model selection for Gaussian regression with random design. Bernoulli10 (2004) 1039–1051. Zbl1064.62030MR2108042
  8. [8] L. Birgé and P. Massart, Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli4 (1998) 329–375. Zbl0954.62033MR1653272
  9. [9] L. Birgé and P. Massart, Minimal penalties for gaussian model selection. Probab. Theory Relat. Fields138 (2006) 33–73. Zbl1112.62082MR2288064
  10. [10] E. Brunel and F. Comte, Penalized contrast estimation of density and hazard rate with censored data. Sankhya67 (2005) 441–475. Zbl1192.62102MR2235573
  11. [11] E. Brunel, F. Comte and A. Guilloux, Nonparametric density estimation in presence of bias and censoring. Test18 (2009) 166–194. Zbl1203.62052MR2495970
  12. [12] T.T. Cai and L.D. Brown, Wavelet shrinkage for nonequispaced samples. Ann. Stat.26 (1998) 1783–1799. Zbl0929.62047MR1673278
  13. [13] G. Chagny, Régression: bases déformées et sélection de modèles par pénalisation et méthode de Lepski. Preprint, hal-00519556 v2. 
  14. [14] F. Comte and Y. Rozenholc, A new algorithm for fixed design regression and denoising. Ann. Inst. Stat. Math.56 (2004) 449–473. Zbl1057.62030MR2095013
  15. [15] R.A. DeVore and G. Lorentz, Constructive approximation, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 303. Springer-Verlag, Berlin (1993). Zbl0797.41016MR1261635
  16. [16] D.L. Donoho, I.M. Johnstone, G. Kerkyacharian and D. Picard, Wavelet shrinkage: asymptopia? With discussion and a reply by the authors. J. Roy. Stat. Soc., Ser. B 57 (1995) 301–369. Zbl0827.62035MR1323344
  17. [17] A. Dvoretzky, J. Kiefer and J. Wolfowitz, Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann. Math. Stat.27 (1956) 642–669. Zbl0073.14603MR83864
  18. [18] S. Efromovich, Nonparametric curve estimation: Methods, theory, and applications. Springer Series in Statistics, Springer-Verlag, New York (1999) xiv+411 Zbl0935.62039MR1705298
  19. [19] J. Fan and I. Gijbels, Variable bandwidth and local linear regression smoothers. Ann. Stat.20 (1992) 2008–2036. Zbl0765.62040MR1193323
  20. [20] S. Gaïffas, On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM: PS 11 (2007) 344–364. Zbl1187.62074MR2339297
  21. [21] A. Goldenshluger and O. Lepski, Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality. Ann. Stat.39 (2011) 1608–1632. Zbl1234.62035MR2850214
  22. [22] G.K. Golubev and M. Nussbaum, Adaptive spline estimates in a nonparametric regression model. Teor. Veroyatnost. i Primenen. ( Russian) 37 (1992) 554–561; translation in Theor. Probab. Appl. 37 (1992) 521–529. Zbl0787.62044MR1214361
  23. [23] W. Härdle and A. Tsybakov, Local polynomial estimators of the volatility function in nonparametric autoregression. J. Econ.81 (1997) 223–242. Zbl0904.62047MR1484586
  24. [24] G. Kerkyacharian and D. Picard, Regression in random design and warped wavelets. Bernoulli10 (2004) 1053–1105. Zbl1067.62039MR2108043
  25. [25] T. Klein and E. Rio, Concentration around the mean for maxima of empirical processes. Ann. Probab.33 (2005) 1060–1077. Zbl1066.60023MR2135312
  26. [26] M. Köhler and A. Krzyzak, Nonparametric regression estimation using penalized least squares. IEEE Trans. Inf. Theory47 (2001) 3054–3058. Zbl1008.62580MR1872867
  27. [27] C. Lacour, Adaptive estimation of the transition density of a particular hidden Markov chain. J. Multivar. Anal.99 (2008) 787–814. Zbl1286.62071MR2405092
  28. [28] E. Nadaraya, On estimating regression. Theory Probab. Appl.9 (1964) 141–142. Zbl0136.40902
  29. [29] T.-M. Pham Ngoc, Regression in random design and Bayesian warped wavelets estimators. Electron. J. Stat.3 (2009) 1084–1112. Zbl1326.62077MR2566182
  30. [30] A.B. Tsybakov, Introduction à l’estimation non-paramétrique, Mathématiques & Applications (Berlin), vol. 41. Springer-Verlag, Berlin (2004). Zbl1029.62034MR2013911
  31. [31] G.S. Watson, Smooth regression analysis. Sankhya A26 (1964) 359–372. Zbl0137.13002MR185765
  32. [32] M. Wegkamp, Model selection in nonparametric regression. Ann. Stat.31 (2003) 252–273. Zbl1019.62037MR1962506

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.