# High-dimensional gaussian model selection on a gaussian design

Annales de l'I.H.P. Probabilités et statistiques (2010)

- Volume: 46, Issue: 2, page 480-524
- ISSN: 0246-0203

## Access Full Article

top## Abstract

top## How to cite

topVerzelen, Nicolas. "High-dimensional gaussian model selection on a gaussian design." Annales de l'I.H.P. Probabilités et statistiques 46.2 (2010): 480-524. <http://eudml.org/doc/242316>.

@article{Verzelen2010,

abstract = {We consider the problem of estimating the conditional mean of a real gaussian variable Y=∑i=1pθiXi+ɛ where the vector of the covariates (Xi)1≤i≤p follows a joint gaussian distribution. This issue often occurs when one aims at estimating the graph or the distribution of a gaussian graphical model. We introduce a general model selection procedure which is based on the minimization of a penalized least squares type criterion. It handles a variety of problems such as ordered and complete variable selection, allows to incorporate some prior knowledge on the model and applies when the number of covariates p is larger than the number of observations n. Moreover, it is shown to achieve a non-asymptotic oracle inequality independently of the correlation structure of the covariates. We also exhibit various minimax rates of estimation in the considered framework and hence derive adaptivity properties of our procedure.},

author = {Verzelen, Nicolas},

journal = {Annales de l'I.H.P. Probabilités et statistiques},

keywords = {model selection; linear regression; oracle inequalities; gaussian graphical models; minimax rates of estimation; Gaussian graphical models},

language = {eng},

number = {2},

pages = {480-524},

publisher = {Gauthier-Villars},

title = {High-dimensional gaussian model selection on a gaussian design},

url = {http://eudml.org/doc/242316},

volume = {46},

year = {2010},

}

TY - JOUR

AU - Verzelen, Nicolas

TI - High-dimensional gaussian model selection on a gaussian design

JO - Annales de l'I.H.P. Probabilités et statistiques

PY - 2010

PB - Gauthier-Villars

VL - 46

IS - 2

SP - 480

EP - 524

AB - We consider the problem of estimating the conditional mean of a real gaussian variable Y=∑i=1pθiXi+ɛ where the vector of the covariates (Xi)1≤i≤p follows a joint gaussian distribution. This issue often occurs when one aims at estimating the graph or the distribution of a gaussian graphical model. We introduce a general model selection procedure which is based on the minimization of a penalized least squares type criterion. It handles a variety of problems such as ordered and complete variable selection, allows to incorporate some prior knowledge on the model and applies when the number of covariates p is larger than the number of observations n. Moreover, it is shown to achieve a non-asymptotic oracle inequality independently of the correlation structure of the covariates. We also exhibit various minimax rates of estimation in the considered framework and hence derive adaptivity properties of our procedure.

LA - eng

KW - model selection; linear regression; oracle inequalities; gaussian graphical models; minimax rates of estimation; Gaussian graphical models

UR - http://eudml.org/doc/242316

ER -

## References

top- [1] H. Akaike. Statistical predictor identification. Ann. Inst. Statist. Math. 22 (1970) 203–217. Zbl0259.62076MR286233
- [2] H. Akaike. A new look at the statistical model identification. IEEE Trans. Automat. Control 19 (1974) 716–723. Zbl0314.62039MR423716
- [3] S. Arlot. Model selection by resampling penalization. Electron. J. Stat. 3 (2009) 557–624. Zbl1326.62097MR2519533
- [4] Y. Baraud, C. Giraud and S. Huet. Gaussian model selection with an unknown variance. Ann. Statist. 37 (2009) 630–672. Zbl1162.62051MR2502646
- [5] P. Bickel, Y. Ritov and A. Tsybakov. Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist. 37 (2009) 1705–1732. Zbl1173.62022MR2533469
- [6] L. Birgé. A new lower bound for multiple hypothesis testing. IEEE Trans. Inform. Theory 51 (2005) 1611–1615. Zbl1283.62030MR2241522
- [7] L. Birgé and P. Massart. Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli 4 (1998) 329–375. Zbl0954.62033MR1653272
- [8] L. Birgé and P. Massart. Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 (2001) 203–268. Zbl1037.62001MR1848946
- [9] L. Birgé and P. Massart. Minimal penalties for Gaussian model selection. Probab. Theory Related Fields 138 (2007) 33–73. Zbl1112.62082MR2288064
- [10] F. Bunea, A. Tsybakov and M. Wegkamp. Aggregation for Gaussian regression. Ann. Statist. 35 (2007) 1674–1697. Zbl1209.62065MR2351101
- [11] F. Bunea, A. Tsybakov and M. Wegkamp. Sparsity oracle inequalities for the Lasso. Electron. J. Stat. 1 (2007) 169–194 (electronic). Zbl1146.62028MR2312149
- [12] E. J. Candes and T. Tao. Decoding by linear programming. IEEE Trans. Inform. Theory 51 (2005) 4203–4215. Zbl1264.94121MR2243152
- [13] E. Candes and T. Tao. The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35 (2007) 2313–2351. Zbl1139.62019MR2382644
- [14] E. Candès and Y. Plan. Near-ideal model selection by l1 minimization. Ann. Statist. To appear, 2009. Zbl1173.62053MR2543688
- [15] R. G. Cowell, A. P. Dawid, S. L. Lauritzen and D. J. Spiegelhalter. Probabilistic Networks and Expert Systems. Statistics for Engineering and Information Science. Springer, New York, 1999. Zbl1120.68444MR1697175
- [16] N. A. C. Cressie. Statistics for Spatial Data. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Wiley, New York, 1993. (Revised reprint of the 1991 edition, Wiley.) Zbl0799.62002MR1239641
- [17] K. R. Davidson and S. J. Szarek. Local operator theory, random matrices and Banach spaces. In Handbook of the Geometry of Banach Spaces, Vol. I 317–366. North-Holland, Amsterdam, 2001. Zbl1067.46008MR1863696
- [18] C. Giraud. Estimation of Gaussian graphs by model selection. Electron. J. Stat. 2 (2008) 542–563. Zbl1320.62094MR2417393
- [19] T. Gneiting. Power-law correlations, related models for long-range dependence and their simulation. J. Appl. Probab. 37 (2000) 1104–1109. Zbl0972.62079MR1808873
- [20] M. Kalisch and P. Bühlmann. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J. Mach. Learn. Res. 8 (2007) 613–636. Zbl1222.68229
- [21] B. Laurent and P. Massart. Adaptive estimation of a quadratic functional by model selection. Ann. Statist. 28 (2000) 1302–1338. Zbl1105.62328MR1805785
- [22] S. L. Lauritzen. Graphical Models. Oxford Statistical Science Series 17. The Clarendon Press, Oxford University Press, New York, 1996. Zbl0907.62001MR1419991
- [23] C. L. Mallows. Some comments on Cp. Technometrics 15 (1973) 661–675. Zbl0269.62061
- [24] P. Massart. Concentration Inequalities and Model Selection. Lecture Notes in Mathematics 1896. Springer, Berlin, 2007. (Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003, with a foreword by Jean Picard.) Zbl1170.60006MR2319879
- [25] N. Meinshausen and P. Bühlmann. High-dimensional graphs and variable selection with the Lasso. Ann. Statist. 34 (2006) 1436–1462. Zbl1113.62082MR2278363
- [26] V. H. de la Peña and E. Giné. Decoupling. Probability and Its Applications. Springer, New York, 1999. (From dependence to independence, randomly stopped processes. U-statistics and processes. Martingales and beyond.) Zbl0918.60021MR1666908
- [27] D. von Rosen. Moments for the inverted Wishart distribution. Scand. J. Statist. 15 (1988) 97–109. Zbl0663.62063MR968156
- [28] H. Rue and L. Held. Gaussian Markov Random Fields: Theory and Applications. Monographs on Statistics and Applied Probability 104. Chapman & Hall/CRC, London, 2005. Zbl1093.60003MR2130347
- [29] K. Sachs, O. Perez, D. Pe’er, D. A. Lauffenburger and G. P. Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science 308 (2005) 523–529.
- [30] J. Schäfer and K. Strimmer. An empirical Bayes approach to inferring large-scale gene association network. Bioinformatics 21 (2005) 754–764.
- [31] G. Schwarz. Estimating the dimension of a model. Ann. Statist. 6 (1978) 461–464. Zbl0379.62005MR468014
- [32] R. Shibata. An optimal selection of regression variables. Biometrika 68 (1981) 45–54. Zbl0464.62054MR614940
- [33] C. Stone. An asymptotically optimal histogram selection rule. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983) 513–520. Wadsworth Statist./Probab. Ser. Wadsworth, Belmont, CA, 1985. MR822050
- [34] R. Tibshirani. Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 (1996) 267–288. Zbl0850.62538MR1379242
- [35] A. Tsybakov. Optimal rates of aggregation. In 16th Annual Conference on Learning Theory 2777 303–313. Springer, Heidelberg, 2003. Zbl1208.62073
- [36] N. Verzelen and F. Villers. Goodness-of-fit tests for high-dimensional Gaussian linear models. Ann. Statist. To appear, 2009. Zbl1183.62074MR2604699
- [37] M. J. Wainwright. Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting. Technical Report 725, Department of Statistics, UC Berkeley, 2007.
- [38] A. Wille, P. Zimmermann, E. Vranova, A. Fürholz, O. Laule, S. Bleuler, L. Hennig, A. Prelic, P. von Rohr, L. Thiele, E. Zitzler, W. Gruissem and P. Bühlmann. Sparse graphical Gaussian modelling of the isoprenoid gene network in arabidopsis thaliana. Genome Biology 5 (2004), no. R92.
- [39] P. Zhao and B. Yu. On model selection consistency of Lasso. J. Mach. Learn. Res. 7 (2006) 2541–2563. Zbl1222.62008MR2274449
- [40] H. Zou. The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc. 101 (2006) 1418–1429. Zbl1171.62326MR2279469

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.