# Model selection for regression on a random design

ESAIM: Probability and Statistics (2010)

- Volume: 6, page 127-146
- ISSN: 1292-8100

## Access Full Article

top## Abstract

top## How to cite

topBaraud, Yannick. "Model selection for regression on a random design." ESAIM: Probability and Statistics 6 (2010): 127-146. <http://eudml.org/doc/104283>.

@article{Baraud2010,

abstract = {
We consider the problem of estimating an unknown regression function
when the design is random with values in $\mathbb\{R\}^k$. Our estimation
procedure is based on model selection and does not rely on any prior
information on the target function. We start with a collection of
linear functional spaces and build, on a data selected space among
this collection, the least-squares estimator. We study the
performance of an estimator which is obtained by modifying this
least-squares estimator on a set of small probability. For the
so-defined estimator, we establish nonasymptotic risk bounds that
can be related to oracle inequalities. As a consequence of these, we
show that our estimator possesses adaptive properties in the
minimax sense over large families of Besov balls Bα,l,∞(R) with R>0, l ≥ 1 and α > α1
where α1 is a positive number satisfying
1/l - 1/2 ≤ α1 < 1/l. We also study the particular case where
the regression function is additive and then obtain an additive
estimator which converges at the same rate as it does when k=1.
},

author = {Baraud, Yannick},

journal = {ESAIM: Probability and Statistics},

keywords = {Nonparametric regression; least-squares
estimators; penalized criteria; minimax rates; Besov spaces; model
selection; adaptive estimation.; least-squares estimators; model selection; adaptive estimation},

language = {eng},

month = {3},

pages = {127-146},

publisher = {EDP Sciences},

title = {Model selection for regression on a random design},

url = {http://eudml.org/doc/104283},

volume = {6},

year = {2010},

}

TY - JOUR

AU - Baraud, Yannick

TI - Model selection for regression on a random design

JO - ESAIM: Probability and Statistics

DA - 2010/3//

PB - EDP Sciences

VL - 6

SP - 127

EP - 146

AB -
We consider the problem of estimating an unknown regression function
when the design is random with values in $\mathbb{R}^k$. Our estimation
procedure is based on model selection and does not rely on any prior
information on the target function. We start with a collection of
linear functional spaces and build, on a data selected space among
this collection, the least-squares estimator. We study the
performance of an estimator which is obtained by modifying this
least-squares estimator on a set of small probability. For the
so-defined estimator, we establish nonasymptotic risk bounds that
can be related to oracle inequalities. As a consequence of these, we
show that our estimator possesses adaptive properties in the
minimax sense over large families of Besov balls Bα,l,∞(R) with R>0, l ≥ 1 and α > α1
where α1 is a positive number satisfying
1/l - 1/2 ≤ α1 < 1/l. We also study the particular case where
the regression function is additive and then obtain an additive
estimator which converges at the same rate as it does when k=1.

LA - eng

KW - Nonparametric regression; least-squares
estimators; penalized criteria; minimax rates; Besov spaces; model
selection; adaptive estimation.; least-squares estimators; model selection; adaptive estimation

UR - http://eudml.org/doc/104283

ER -

## References

top- Y. Baraud, Model selection for regression on a fixed design. Probab. Theory Related Fields117 (2000) 467-493.
- A. Barron, L. Birgé and P. Massart, Risk bounds for model selection via penalization. Probab. Theory Related Fields113 (1999) 301-413.
- A.R. Barron and T.M. Cover, Minimum complexity density estimation. IEEE Trans. Inform. Theory37 (1991) 1738.
- L. Birgé and P. Massart, An adaptive compression algorithm in Besov spaces. Constr. Approx.16 (2000) 1-36.
- L. Birgé and P. Massart, Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli4 (1998) 329-375.
- L. Birgé and P. Massart, Gaussian model selection. JEMS3 (2001) 203-268.
- L. Birgé and Massart, A generalized Cp criterion for Gaussian model selection, Technical Report. University Paris 6, PMA-647 (2001).
- L. Birgé and Y. Rozenholc, How many bins should be put in a regular histogram, Technical Report. University Paris 6, PMA-721 (2002).
- O. Catoni, Statistical learning theory and stochastic optimization, in École d'été de probabilités de Saint-Flour. Springer (2001).
- A. Cohen, I. Daubechies and P. Vial, Wavelet and fast wavelet transform on an interval. Appl. Comp. Harmon. Anal.1 (1993) 54-81.
- I. Daubechies, Ten lectures on wavelets. SIAM: Philadelphia (1992).
- R.A. DeVore and G.G. Lorentz, Constructive approximation. Springer-Verlag, Berlin (1993).
- D.L. Donoho and I.M. Johnstone, Ideal spatial adaptation via wavelet shrinkage. Biometrika81 (1994) 425-455.
- D.L. Donoho and I.M. Johnstone, Minimax estimation via wavelet shrinkage. Ann. Statist.26 (1998) 879-921.
- M. Kohler, Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. J. Statist. Plann. Inference89 (2000) 1-23.
- M. Kohler, Nonparametric regression function estimation using interaction least square splines and complexity regularization. Metrika47 (1998) 147-163.
- A.P. Korostelev and A.B. Tsybakov, Minimax theory of image reconstruction. Springer-Verlag, New York NY, Lecture Notes in Statis. (1993).
- C.J. Stone, Additive regression and other nonparametric models. Ann. Statist.13 (1985) 689-705.
- M. Wegkamp, Model selection in non-parametric regression, Preprint. Yale University (2000).
- Y. Yang, Model selection for nonparametric regression. Statist. Sinica9 (1999) 475-499.
- Y. Yang, Combining different procedures for adaptive regression. J. Multivariate Anal.74 (2000) 135-161.
- Y. Yang and A. Barron, Information-Theoretic determination of minimax rates of convergence. Ann. Statist.27 (1999) 1564-1599.

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.