# Penalization versus Goldenshluger − Lepski strategies in warped bases regression

ESAIM: Probability and Statistics (2013)

- Volume: 17, page 328-358
- ISSN: 1292-8100

## Access Full Article

top## Abstract

top## How to cite

topChagny, Gaëlle. "Penalization versus Goldenshluger − Lepski strategies in warped bases regression." ESAIM: Probability and Statistics 17 (2013): 328-358. <http://eudml.org/doc/273619>.

@article{Chagny2013,

abstract = {This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, $\hat\{f\}_l=\hat\{h\}_l\circ G$f̂l = ĥl°G if Gis known, or $\hat\{f\}_l=\hat\{h\}_l\circ \hat\{G\}$f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.},

author = {Chagny, Gaëlle},

journal = {ESAIM: Probability and Statistics},

keywords = {adaptive estimator; model selection; nonparametric regression estimation; warped bases},

language = {eng},

pages = {328-358},

publisher = {EDP-Sciences},

title = {Penalization versus Goldenshluger − Lepski strategies in warped bases regression},

url = {http://eudml.org/doc/273619},

volume = {17},

year = {2013},

}

TY - JOUR

AU - Chagny, Gaëlle

TI - Penalization versus Goldenshluger − Lepski strategies in warped bases regression

JO - ESAIM: Probability and Statistics

PY - 2013

PB - EDP-Sciences

VL - 17

SP - 328

EP - 358

AB - This paper deals with the problem of estimating a regression function f, in a random design framework. We build and study two adaptive estimators based on model selection, applied with warped bases. We start with a collection of finite dimensional linear spaces, spanned by orthonormal bases. Instead of expanding directly the target function f on these bases, we rather consider the expansion of h = f ∘ G-1, where G is the cumulative distribution function of the design, following Kerkyacharian and Picard [Bernoulli 10 (2004) 1053–1105]. The data-driven selection of the (best) space is done with two strategies: we use both a penalization version of a “warped contrast”, and a model selection device in the spirit of Goldenshluger and Lepski [Ann. Stat. 39 (2011) 1608–1632]. We propose by these methods two functions, ĥl (l = 1, 2), easier to compute than least-squares estimators. We establish nonasymptotic mean-squared integrated risk bounds for the resulting estimators, $\hat{f}_l=\hat{h}_l\circ G$f̂l = ĥl°G if Gis known, or $\hat{f}_l=\hat{h}_l\circ \hat{G}$f̂l = ĥl°Ĝ (l = 1,2) otherwise, where Ĝ is the empirical distribution function. We study also adaptive properties, in case the regression function belongs to a Besov or Sobolev space, and compare the theoretical and practical performances of the two selection rules.

LA - eng

KW - adaptive estimator; model selection; nonparametric regression estimation; warped bases

UR - http://eudml.org/doc/273619

ER -

## References

top- [1] A. Antoniadis, G. Grégoire and P. Vial, Random design wavelet curve smoothing. Statist. Probab. Lett.35 (1997) 225–232. Zbl0889.62029MR1484959
- [2] J.Y. Audibert and O. Catoni, Robust linear least squares regression. Ann. Stat. (2011) (to appear), arXiv:1010.0074. Zbl1231.62126MR2906886
- [3] J.Y. Audibert and O. Catoni, Robust linear regression through PAC-Bayesian truncation. Preprint, arXiv:1010.0072.
- [4] Y. Baraud, Model selection for regression on a random design. ESAIM: PS 6 (2002) 127–146. Zbl1059.62038MR1918295
- [5] A. Barron, L. Birgé and P. Massart, Risk bounds for model selection via penalization. Probab. Theory Relat. Fields113 (1999) 301–413. Zbl0946.62036MR1679028
- [6] J.P. Baudry, C. Maugis and B. Michel, Slope heuristics: overview and implementation. Stat. Comput. 22-2 (2011) 455–470. Zbl1322.62007MR2865029
- [7] L. Birgé, Model selection for Gaussian regression with random design. Bernoulli10 (2004) 1039–1051. Zbl1064.62030MR2108042
- [8] L. Birgé and P. Massart, Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli4 (1998) 329–375. Zbl0954.62033MR1653272
- [9] L. Birgé and P. Massart, Minimal penalties for gaussian model selection. Probab. Theory Relat. Fields138 (2006) 33–73. Zbl1112.62082MR2288064
- [10] E. Brunel and F. Comte, Penalized contrast estimation of density and hazard rate with censored data. Sankhya67 (2005) 441–475. Zbl1192.62102MR2235573
- [11] E. Brunel, F. Comte and A. Guilloux, Nonparametric density estimation in presence of bias and censoring. Test18 (2009) 166–194. Zbl1203.62052MR2495970
- [12] T.T. Cai and L.D. Brown, Wavelet shrinkage for nonequispaced samples. Ann. Stat.26 (1998) 1783–1799. Zbl0929.62047MR1673278
- [13] G. Chagny, Régression: bases déformées et sélection de modèles par pénalisation et méthode de Lepski. Preprint, hal-00519556 v2.
- [14] F. Comte and Y. Rozenholc, A new algorithm for fixed design regression and denoising. Ann. Inst. Stat. Math.56 (2004) 449–473. Zbl1057.62030MR2095013
- [15] R.A. DeVore and G. Lorentz, Constructive approximation, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 303. Springer-Verlag, Berlin (1993). Zbl0797.41016MR1261635
- [16] D.L. Donoho, I.M. Johnstone, G. Kerkyacharian and D. Picard, Wavelet shrinkage: asymptopia? With discussion and a reply by the authors. J. Roy. Stat. Soc., Ser. B 57 (1995) 301–369. Zbl0827.62035MR1323344
- [17] A. Dvoretzky, J. Kiefer and J. Wolfowitz, Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann. Math. Stat.27 (1956) 642–669. Zbl0073.14603MR83864
- [18] S. Efromovich, Nonparametric curve estimation: Methods, theory, and applications. Springer Series in Statistics, Springer-Verlag, New York (1999) xiv+411 Zbl0935.62039MR1705298
- [19] J. Fan and I. Gijbels, Variable bandwidth and local linear regression smoothers. Ann. Stat.20 (1992) 2008–2036. Zbl0765.62040MR1193323
- [20] S. Gaïffas, On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM: PS 11 (2007) 344–364. Zbl1187.62074MR2339297
- [21] A. Goldenshluger and O. Lepski, Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality. Ann. Stat.39 (2011) 1608–1632. Zbl1234.62035MR2850214
- [22] G.K. Golubev and M. Nussbaum, Adaptive spline estimates in a nonparametric regression model. Teor. Veroyatnost. i Primenen. ( Russian) 37 (1992) 554–561; translation in Theor. Probab. Appl. 37 (1992) 521–529. Zbl0787.62044MR1214361
- [23] W. Härdle and A. Tsybakov, Local polynomial estimators of the volatility function in nonparametric autoregression. J. Econ.81 (1997) 223–242. Zbl0904.62047MR1484586
- [24] G. Kerkyacharian and D. Picard, Regression in random design and warped wavelets. Bernoulli10 (2004) 1053–1105. Zbl1067.62039MR2108043
- [25] T. Klein and E. Rio, Concentration around the mean for maxima of empirical processes. Ann. Probab.33 (2005) 1060–1077. Zbl1066.60023MR2135312
- [26] M. Köhler and A. Krzyzak, Nonparametric regression estimation using penalized least squares. IEEE Trans. Inf. Theory47 (2001) 3054–3058. Zbl1008.62580MR1872867
- [27] C. Lacour, Adaptive estimation of the transition density of a particular hidden Markov chain. J. Multivar. Anal.99 (2008) 787–814. Zbl1286.62071MR2405092
- [28] E. Nadaraya, On estimating regression. Theory Probab. Appl.9 (1964) 141–142. Zbl0136.40902
- [29] T.-M. Pham Ngoc, Regression in random design and Bayesian warped wavelets estimators. Electron. J. Stat.3 (2009) 1084–1112. Zbl1326.62077MR2566182
- [30] A.B. Tsybakov, Introduction à l’estimation non-paramétrique, Mathématiques & Applications (Berlin), vol. 41. Springer-Verlag, Berlin (2004). Zbl1029.62034MR2013911
- [31] G.S. Watson, Smooth regression analysis. Sankhya A26 (1964) 359–372. Zbl0137.13002MR185765
- [32] M. Wegkamp, Model selection in nonparametric regression. Ann. Stat.31 (2003) 252–273. Zbl1019.62037MR1962506

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.