# Risk bounds for mixture density estimation

Alexander Rakhlin; Dmitry Panchenko; Sayan Mukherjee

ESAIM: Probability and Statistics (2010)

- Volume: 9, page 220-229
- ISSN: 1292-8100

## Access Full Article

top## Abstract

top## How to cite

topRakhlin, Alexander, Panchenko, Dmitry, and Mukherjee, Sayan. "Risk bounds for mixture density estimation." ESAIM: Probability and Statistics 9 (2010): 220-229. <http://eudml.org/doc/104333>.

@article{Rakhlin2010,

abstract = {
In this paper we focus on the problem of estimating a bounded
density using a finite combination of densities from a given
class. We consider the Maximum Likelihood Estimator (MLE) and the
greedy procedure described by Li and Barron (1999)
under the additional assumption of boundedness of densities. We
prove an $O(\frac\{1\}\{\sqrt\{n\}\})$ bound on the estimation error
which does not depend on the number of densities in the estimated
combination. Under the boundedness assumption,
this improves the bound of Li and Barron by removing the $\log n$
factor and also generalizes it to the base classes with converging
Dudley integral.
},

author = {Rakhlin, Alexander, Panchenko, Dmitry, Mukherjee, Sayan},

journal = {ESAIM: Probability and Statistics},

keywords = {Mixture density estimation; maximum likelihood; Rademacher processes.; Rademacher processes},

language = {eng},

month = {3},

pages = {220-229},

publisher = {EDP Sciences},

title = {Risk bounds for mixture density estimation},

url = {http://eudml.org/doc/104333},

volume = {9},

year = {2010},

}

TY - JOUR

AU - Rakhlin, Alexander

AU - Panchenko, Dmitry

AU - Mukherjee, Sayan

TI - Risk bounds for mixture density estimation

JO - ESAIM: Probability and Statistics

DA - 2010/3//

PB - EDP Sciences

VL - 9

SP - 220

EP - 229

AB -
In this paper we focus on the problem of estimating a bounded
density using a finite combination of densities from a given
class. We consider the Maximum Likelihood Estimator (MLE) and the
greedy procedure described by Li and Barron (1999)
under the additional assumption of boundedness of densities. We
prove an $O(\frac{1}{\sqrt{n}})$ bound on the estimation error
which does not depend on the number of densities in the estimated
combination. Under the boundedness assumption,
this improves the bound of Li and Barron by removing the $\log n$
factor and also generalizes it to the base classes with converging
Dudley integral.

LA - eng

KW - Mixture density estimation; maximum likelihood; Rademacher processes.; Rademacher processes

UR - http://eudml.org/doc/104333

ER -

## References

top- A.R. Barron, Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory39 (1993) 930–945.
- A.R. Barron, Approximation and estimation bounds for artificial neural networks. Machine Learning14 (1994) 115–133.
- L. Birgé and P. Massart, Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields97 (1993) 113–150.
- R.M. Dudley, Uniform Central Limit Theorems. Cambridge University Press (1999).
- L.K. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for Projection Pursuit Regression and neural network training. Ann. Stat.20 (1992) 608–613.
- M. Ledoux and M. Talagrand, Probability in Banach Spaces. Springer-Verlag, New York (1991).
- J. Li and A. Barron, Mixture density estimation, in Advances in Neural information processings systems 12, S.A. Solla, T.K. Leen and K.-R. Muller Ed. San Mateo, CA. Morgan Kaufmann Publishers (1999).
- J. Li, Estimation of Mixture Models. Ph.D. Thesis, The Department of Statistics. Yale University (1999).
- C. McDiarmid, On the method of bounded differences. Surveys in Combinatorics (1989) 148–188.
- S. Mendelson, On the size of convex hulls of small sets. J. Machine Learning Research2 (2001) 1–18.
- P. Niyogi and F. Girosi, Generalization bounds for function approximation from scattered noisy data. Adv. Comput. Math.10 (1999) 51–80.
- S.A. van de Geer, Rates of convergence for the maximum likelihood estimator in mixture models. Nonparametric Statistics6 (1996) 293–310.
- S.A. van de Geer, Empirical Processes in M-Estimation. Cambridge University Press (2000).
- A.W. van der Vaart and J.A. Wellner, Weak Convergence and Empirical Processes with Applications to Statistics. Springer-Verlag, New York (1996).
- W.H. Wong and X. Shen, Probability inequalities for likelihood ratios and convergence rates for sieve mles. Ann. Stat.23 (1995) 339–362.

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.