# On EM algorithms and their proximal generalizations

Stéphane Chrétien; Alfred O. Hero

ESAIM: Probability and Statistics (2008)

- Volume: 12, page 308-326
- ISSN: 1292-8100

## Access Full Article

top## Abstract

top## How to cite

topChrétien, Stéphane, and Hero, Alfred O.. "On EM algorithms and their proximal generalizations." ESAIM: Probability and Statistics 12 (2008): 308-326. <http://eudml.org/doc/250388>.

@article{Chrétien2008,

abstract = {
In this paper, we analyze the celebrated EM algorithm from
the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new
results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.
},

author = {Chrétien, Stéphane, Hero, Alfred O.},

journal = {ESAIM: Probability and Statistics},

keywords = {Maximum Likelihood Estimation (MLE); EM algorithm; proximal point algorithm; Karush-Kuhn-Tucker condition; mixture densities; competing risks models; maximum likelihood estimation (MLE); expectation and maximization (EM) algorithm; proximal point algorithms},

language = {eng},

month = {5},

pages = {308-326},

publisher = {EDP Sciences},

title = {On EM algorithms and their proximal generalizations},

url = {http://eudml.org/doc/250388},

volume = {12},

year = {2008},

}

TY - JOUR

AU - Chrétien, Stéphane

AU - Hero, Alfred O.

TI - On EM algorithms and their proximal generalizations

JO - ESAIM: Probability and Statistics

DA - 2008/5//

PB - EDP Sciences

VL - 12

SP - 308

EP - 326

AB -
In this paper, we analyze the celebrated EM algorithm from
the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new
results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

LA - eng

KW - Maximum Likelihood Estimation (MLE); EM algorithm; proximal point algorithm; Karush-Kuhn-Tucker condition; mixture densities; competing risks models; maximum likelihood estimation (MLE); expectation and maximization (EM) algorithm; proximal point algorithms

UR - http://eudml.org/doc/250388

ER -

## References

top- H. Ahn, H. Moon and R.L. Kodell, Attribution of tumour lethality and estimation of the time to onset of occult tumours in the absence of cause-of-death information. J. Roy. Statist. Soc. Ser. C49 (2000) 157–169.
- M.J. Box, A new method of constrained optimization and a comparison with other methods. Comp. J.8 (1965) 42–52.
- G. Celeux, S. Chretien, F. Forbes and A. Mkhadri, A component-wise EM algorithm for mixtures. J. Comput. Graph. Statist.10 (2001), 697–712 and INRIA RR-3746, Aug. 1999.
- S. Chretien and A.O. Hero, Acceleration of the EM algorithm via proximal point iterations, in Proceedings of the International Symposium on Information Theory, MIT, Cambridge (1998) 444.
- S. Chrétien and A. Hero, Kullback proximal algorithms for maximum-likelihood estimation. IEEE Trans. Inform. Theory46 (2000) 1800–1810.
- I. Csiszár, Information-type measures of divergence of probability distributions and indirect observations. Studia Sci. Math. Hung.2 (1967) 299–318.
- A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc., Ser. B39 (1977) 1–38.
- I.A. Ibragimov and R.Z. Has'minskii, Statistical estimation: Asymptotic theory. Springer-Verlag, New York (1981).
- Journal of Statistical Planning and Inference No. 107 (2002) 1–2.
- A.T. Kalai and S. Vempala, Simulated annealing for convex optimization. Math. Oper. Res.31 (2006) 253–266.
- B. Martinet, Régularisation d'inéquation variationnelles par approximations successives. Revue Francaise d'Informatique et de Recherche Operationnelle3 (1970) 154–179.
- G.J. McLachlan and T. Krishnan, The EM algorithm and extensions, Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley and Sons, Inc., New York (1997).
- H. Moon, H. Ahn, R. Kodell and B. Pearce, A comparison of a mixture likelihood method and the EM algorithm for an estimation problme in animal carcinogenicity studies. Comput. Statist. Data Anal.31 (1999) 227–238.
- A.M. Ostrowski, Solution of equations and systems of equations. Pure and Applied Mathematics, Vol. IX. Academic Press, New York-London (1966).
- R.T. Rockafellar, Monotone operators and the proximal point algorithm. SIAM J. Control Optim.14 (1976) 877–898.
- M. Teboulle, Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res.17 (1992) 670–690.
- P. Tseng, An analysis of the EM algorithm and entropy-like proximal point methods. Math. Oper. Res.29 (2004) 27–44.
- C.F.J. Wu, On the convergence properties of the EM algorithm. Ann. Stat.11 (1983) 95–103.
- Z.B. Zabinsky, Stochastic adaptive search for global optimization. Nonconvex Optimization and its Applications72. Kluwer Academic Publishers, Boston, MA (2003).
- W.I. Zangwill and B. Mond, Nonlinear programming: a unified approach. Prentice-Hall International Series in Management. Prentice-Hall, Inc., Englewood Cliffs, N.J. (1969).

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.