# Model selection and estimation of a component in additive regression

ESAIM: Probability and Statistics (2014)

- Volume: 18, page 77-116
- ISSN: 1292-8100

## Access Full Article

top## Abstract

top## How to cite

topGendre, Xavier. "Model selection and estimation of a component in additive regression." ESAIM: Probability and Statistics 18 (2014): 77-116. <http://eudml.org/doc/274383>.

@article{Gendre2014,

abstract = {Let Y ∈ ℝn be a random vector with mean s and covariance matrix σ2PntPn where Pn is some known n × n-matrix. We construct a statistical procedure to estimate s as well as under moment condition on Y or Gaussian hypothesis. Both cases are developed for known or unknown σ2. Our approach is free from any prior assumption on s and is based on non-asymptotic model selection methods. Given some linear spaces collection \{Sm, m ∈ ℳ\}, we consider, for any m ∈ ℳ, the least-squares estimator ŝm of s in Sm. Considering a penalty function that is not linear in the dimensions of the Sm’s, we select some m̂ ∈ ℳ in order to get an estimator ŝm̂ with a quadratic risk as close as possible to the minimal one among the risks of the ŝm’s. Non-asymptotic oracle-type inequalities and minimax convergence rates are proved for ŝm̂. A special attention is given to the estimation of a non-parametric component in additive models. Finally, we carry out a simulation study in order to illustrate the performances of our estimators in practice.},

author = {Gendre, Xavier},

journal = {ESAIM: Probability and Statistics},

keywords = {model selection; nonparametric regression; penalized criterion; oracle inequality; correlated data; additive regression; minimax rate},

language = {eng},

pages = {77-116},

publisher = {EDP-Sciences},

title = {Model selection and estimation of a component in additive regression},

url = {http://eudml.org/doc/274383},

volume = {18},

year = {2014},

}

TY - JOUR

AU - Gendre, Xavier

TI - Model selection and estimation of a component in additive regression

JO - ESAIM: Probability and Statistics

PY - 2014

PB - EDP-Sciences

VL - 18

SP - 77

EP - 116

AB - Let Y ∈ ℝn be a random vector with mean s and covariance matrix σ2PntPn where Pn is some known n × n-matrix. We construct a statistical procedure to estimate s as well as under moment condition on Y or Gaussian hypothesis. Both cases are developed for known or unknown σ2. Our approach is free from any prior assumption on s and is based on non-asymptotic model selection methods. Given some linear spaces collection {Sm, m ∈ ℳ}, we consider, for any m ∈ ℳ, the least-squares estimator ŝm of s in Sm. Considering a penalty function that is not linear in the dimensions of the Sm’s, we select some m̂ ∈ ℳ in order to get an estimator ŝm̂ with a quadratic risk as close as possible to the minimal one among the risks of the ŝm’s. Non-asymptotic oracle-type inequalities and minimax convergence rates are proved for ŝm̂. A special attention is given to the estimation of a non-parametric component in additive models. Finally, we carry out a simulation study in order to illustrate the performances of our estimators in practice.

LA - eng

KW - model selection; nonparametric regression; penalized criterion; oracle inequality; correlated data; additive regression; minimax rate

UR - http://eudml.org/doc/274383

ER -

## References

top- [1] H. Akaike, Statistical predictor identification. Ann. Inst. Stat. Math.22 (1970) 203–217. Zbl0259.62076MR286233
- [2] S. Arlot, Choosing a penalty for model selection in heteroscedastic regression. Preprint arXiv:0812.3141v2 (2010).
- [3] S. Arlot and P. Massart, Data-driven calibration of penalties for least-squares regression. J. Machine Learn. Research10 (2009) 245–279.
- [4] Y. Baraud, Model selection for regression on a fixed design. Probab. Theory Related Fields117 (2000) 467–493. Zbl0997.62027MR1777129
- [5] Y. Baraud, Model selection for regression on a random design. ESAIM: Probab. Statist. 6 (2002) 127–146. Zbl1059.62038MR1918295
- [6] Y. Baraud, F. Comte and G. Viennet, Adaptive estimation in autoregression or β-mixing regression via model selection. Ann. Stat.29 (2001) 839–875. Zbl1012.62034MR1865343
- [7] L. Birgé and P. Massart, From model selection to adaptive estimation. Festschrift for Lucien Lecam: Research Papers in Probab. Stat. (1997) 55–87. Zbl0920.62042MR1462939
- [8] L. Birgé and P. Massart, Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli4 (1998) 329–375. Zbl0954.62033MR1653272
- [9] L. Birgé and P. Massart, An adaptive compression algorithm in Besov spaces. Constr. Approx.16 (2000) 1–36. Zbl1004.41006MR1848840
- [10] L. Birgé and P. Massart. Gaussian model selection. J. Europ. Math. Soc.3 (2001) 203–268. Zbl1037.62001MR1848946
- [11] L. Birgé and P. Massart, Minimal penalties for gaussian model selection. Probab. Theory Related Fields138 (2007) 33–73. Zbl1112.62082MR2288064
- [12] L. Breiman and J.H. Friedman, Estimating optimal transformations for multiple regression and correlations (with discussion). J. Amer. Stat. Assoc.80 (1985) 580–619. Zbl0594.62044MR803258
- [13] E. Brunel and F. Comte, Adaptive nonparametric regression estimation in presence of right censoring. Math. Methods Stat.15 (2006) 233–255. MR2278288
- [14] E. Brunel and F. Comte, Model selection for additive regression models in the presence of censoring, chapt. 1 in “Mathematical Methods in Survival Analysis, Reliability and Quality of Life”, Wiley (2008) 17–31. MR2730701
- [15] A. Buja, T.J. Hastie and R.J. Tibshirani, Linear smoothers and additive models (with discussion). Ann. Stat.17 (1989) 453–555. Zbl0689.62029MR994249
- [16] F. Comte and Y. Rozenholc, Adaptive estimation of mean and volatility functions in (auto-)regressive models. Stoch. Process. Appl.97 (2002) 111–145. Zbl1064.62046MR1870963
- [17] X. Gendre, Simultaneous estimation of the mean and the variance in heteroscedastic gaussian regression. Electron. J. Stat.2 (2008) 1345–1372. Zbl1320.62092MR2471290
- [18] W. Härdle, M. Müller, S. Sperlich and A. Werwatz. Nonparametric and Semiparametric Models. Springer (2004). Zbl1059.62032MR2061786
- [19] T.J. Hastie and R.J. Tibshirani, Generalized additive models. Chapman and Hall (1990). Zbl0747.62061MR1082147
- [20] R.A. Horn and C.R. Johnson, Matrix analysis. Cambridge University Press (1990). Zbl0704.15002MR1084815
- [21] B. Laurent, J.M. Loubes and C. Marteau, Testing inverse problems: a direct or an indirect problem? J. Stat. Plann. Inference141 (2011) 1849–1861. Zbl05862619MR2763215
- [22] B. Laurent and P. Massart, Adaptive estimation of a quadratic functional by model selection. Ann. Stat.28 (2000) 1302–1338. Zbl1105.62328MR1805785
- [23] W. Leontief, Introduction to a theory of the internal structure of functional relationships. Econometrica15 (1947) 361–373. Zbl0030.17105MR25144
- [24] O. Linton and J.P. Nielsen, A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika82 (1995) 93–101. Zbl0823.62036MR1332841
- [25] C.L. Mallows, Some comments on cp. Technometrics 15 (1973) 661–675,. Zbl0269.62061
- [26] E. Mammen, O. Linton and J.P. Nielsen, The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Stat.27 (1999) 1443–1490. Zbl0986.62028MR1742496
- [27] P. Massart, Concentration inequalities and model selection, in vol. 1896 of Lect. Notes Math. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23 (2003). Springer, Berlin (2007). Zbl1170.60006MR2319879
- [28] A.D.R. McQuarrie and C.L. Tsai, Regression and times series model selection. River Edge, NJ (1998). Zbl0907.62095MR1641582
- [29] L. Meier, S. van de Geer and P. Bühlmann, High-dimensional additive modeling. Ann. Stat.37 (2009) 3779–3821. Zbl05644256MR2572443
- [30] J. Opsomer and D. Ruppert, Fitting a bivariate additive model by local polynomial regression. Ann. Stat.25 (1997) 186–211. Zbl0869.62026MR1429922
- [31] V.V. Petrov, Limit theorems of probability theory: sequences of independent random variables. Oxford Studies Probab. 4 (1995). Zbl0826.60001MR1353441
- [32] P.D. Ravikumar, H. Liu, J.D. Lafferty and L.A. Wasserman, Sparse additive models. J. Royal Statist. Soc.71 (2009) 1009–1030. MR2750255
- [33] S. Robin, F. Rodolphe and S. Schbath, DNA, Words and Models. Cambridge University Press (2005). Zbl1185.92047MR2214200
- [34] D. Ruppert and M.P. Wand, Multivariate locally weighted least squares regression. Ann. Stat.22 (1994) 1346–1370. Zbl0821.62020MR1311979
- [35] H. Scheffé, The analysis of variance. Wiley-Interscience (1959). Zbl0998.62500
- [36] E. Severance-Lossin and S. Sperlich, Estimation of derivatives for additive separable models. Statististics33 (1999) 241–265. Zbl0938.62044MR1750253
- [37] C.J. Stone, Additive regression and other nonparametric models. Ann. Stat.14 (1985) 590–606. Zbl0603.62050MR790566
- [38] D. Tjøstheim and B. Auestad, Nonparametric identification of nonlinear time series: Selecting significant lags. J. Amer. Stat. Assoc.89 (1994) 1410–1430. Zbl0813.62037MR1310231
- [39] B. von Bahr and C.G. Esseen, Inequalities for the rth absolute moment of a sum of random variables 1 ≤ r ≤ 2 . Ann. Math. Stat. 36 (1965) 299–303. Zbl0134.36902MR170407

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.