Estimating composite functions by model selection

Yannick Baraud; Lucien Birgé

Annales de l'I.H.P. Probabilités et statistiques (2014)

  • Volume: 50, Issue: 1, page 285-314
  • ISSN: 0246-0203

Abstract

top
We consider the problem of estimating a function on for large values of by looking for some best approximation of by composite functions of the form . Our solution is based on model selection and leads to a very general approach to solve this problem with respect to many different types of functions and statistical frameworks. In particular, we handle the problems of approximating by additive functions, single and multiple index models, artificial neural networks, mixtures of Gaussian densities (when is a density) among other examples. We also investigate the situation where for functions and belonging to possibly anisotropic smoothness classes. In this case, our approach leads to a completely adaptive estimator with respect to the regularities of and .

How to cite

top

Baraud, Yannick, and Birgé, Lucien. "Estimating composite functions by model selection." Annales de l'I.H.P. Probabilités et statistiques 50.1 (2014): 285-314. <http://eudml.org/doc/272057>.

@article{Baraud2014,
abstract = {We consider the problem of estimating a function $s$ on $[-1,1]^\{k\}$ for large values of $k$ by looking for some best approximation of $s$ by composite functions of the form $g\circ u$. Our solution is based on model selection and leads to a very general approach to solve this problem with respect to many different types of functions $g,u$ and statistical frameworks. In particular, we handle the problems of approximating $s$ by additive functions, single and multiple index models, artificial neural networks, mixtures of Gaussian densities (when $s$ is a density) among other examples. We also investigate the situation where $s=g\circ u$ for functions $g$ and $u$ belonging to possibly anisotropic smoothness classes. In this case, our approach leads to a completely adaptive estimator with respect to the regularities of $g$ and $u$.},
author = {Baraud, Yannick, Birgé, Lucien},
journal = {Annales de l'I.H.P. Probabilités et statistiques},
keywords = {curve estimation; model selection; composite functions; adaptation; single index model; artificial neural networks; gaussian mixtures; Gaussian mixtures},
language = {eng},
number = {1},
pages = {285-314},
publisher = {Gauthier-Villars},
title = {Estimating composite functions by model selection},
url = {http://eudml.org/doc/272057},
volume = {50},
year = {2014},
}

TY - JOUR
AU - Baraud, Yannick
AU - Birgé, Lucien
TI - Estimating composite functions by model selection
JO - Annales de l'I.H.P. Probabilités et statistiques
PY - 2014
PB - Gauthier-Villars
VL - 50
IS - 1
SP - 285
EP - 314
AB - We consider the problem of estimating a function $s$ on $[-1,1]^{k}$ for large values of $k$ by looking for some best approximation of $s$ by composite functions of the form $g\circ u$. Our solution is based on model selection and leads to a very general approach to solve this problem with respect to many different types of functions $g,u$ and statistical frameworks. In particular, we handle the problems of approximating $s$ by additive functions, single and multiple index models, artificial neural networks, mixtures of Gaussian densities (when $s$ is a density) among other examples. We also investigate the situation where $s=g\circ u$ for functions $g$ and $u$ belonging to possibly anisotropic smoothness classes. In this case, our approach leads to a completely adaptive estimator with respect to the regularities of $g$ and $u$.
LA - eng
KW - curve estimation; model selection; composite functions; adaptation; single index model; artificial neural networks; gaussian mixtures; Gaussian mixtures
UR - http://eudml.org/doc/272057
ER -

References

top
  1. [1] N. Akakpo. Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Methods Statist.21 (2012) 1–28. Zbl1308.62070MR2901269
  2. [2] Y. Baraud. Estimator selection with respect to Hellinger-type risks. Probab. Theory Related Fields151 (2011) 353–401. Zbl05968717MR2834722
  3. [3] Y. Baraud, F. Comte and G. Viennet. Model selection for (auto-)regression with dependent data. ESAIM Probab. Stat.5 (2001) 33–49. Zbl0990.62035MR1845321
  4. [4] Y. Baraud, C. Giraud and S. Huet. Gaussian model selection with an unknown variance. Ann. Statist.37 (2009) 630–672. Zbl1162.62051MR2502646
  5. [5] A. R. Barron, L. Birgé and P. Massart. Risk bounds for model selection via penalization. Probab. Theory Related Fields113 (1999) 301–413. Zbl0946.62036MR1679028
  6. [6] A. R. Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory39 (1993) 930–945. Zbl0818.68126MR1237720
  7. [7] A. R. Barron. Approximation and estimation bounds for artificial neural networks. Machine Learning14 (1994) 115–133. Zbl0818.68127
  8. [8] L. Birgé. Model selection via testing: An alternative to (penalized) maximum likelihood estimators. Ann. Inst. Henri Poincaré Probab. Stat.42 (2006) 273–325. Zbl1333.62094MR2219712
  9. [9] L. Birgé. Model selection for Poisson processes. In Asymptotics: Particles, Processes and Inverse Problems, Festschrift for Piet Groeneboom 32–64. E. Cator, G. Jongbloed, C. Kraaikamp, R. Lopuhaä and J. Wellner (Eds). IMS Lecture Notes – Monograph Series 55. Inst. Math. Statist., Beachwood, OH, 2007. Zbl1176.62082MR2459930
  10. [10] L. Birgé. Model selection for density estimation with -loss. Probab. Theory Related Fields. To appear. Available at http://arxiv.org/abs/1102.2818. Zbl1285.62037
  11. [11] L. Birgé and P. Massart. Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 (2001) 203–268. Zbl1037.62001MR1848946
  12. [12] W. Dahmen, R. DeVore and K. Scherer. Multidimensional spline approximation. SIAM J. Numer. Anal.17 (1980) 380–402. Zbl0437.41010MR581486
  13. [13] R. DeVore and G. Lorentz. Constructive Approximation. Springer, Berlin, 1993. Zbl0797.41016MR1261635
  14. [14] J. Friedman and J. Tukey. A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. C-23 (1974) 881–890. Zbl0284.68079
  15. [15] R. Hochmuth. Wavelet characterizations for anisotropic Besov spaces. Appl. Comput. Harmon. Anal.12 (2002) 179–208. Zbl1003.42024MR1884234
  16. [16] J. L. Horowitz and E. Mammen. Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions. Ann. Statist.35 (2007) 2589–2619. Zbl1129.62034MR2382659
  17. [17] P. J. Huber. Projection pursuit (with discussion). Ann. Statist.13 (1985) 435–525. Zbl0595.62059MR790553
  18. [18] A. B. Juditsky, O. V. Lepski and A. B. Tsybakov. Nonparametric estimation of composite functions. Ann. Statist.37 (2009) 1360–1404. Zbl1160.62030MR2509077
  19. [19] C. Maugis and B. Michel. A non asymptotic penalized criterion for Gaussian mixture model selection. ESAIM Probab. Stat.15 (2011) 41–68. Zbl06157507MR2870505
  20. [20] C. J. Stone. Optimal global rates of convergence for nonparametric regression. Ann. Statist.10 (1982) 1040–1053. Zbl0511.62048MR673642

NotesEmbed ?

top

You must be logged in to post comments.