On EM algorithms and their proximal generalizations

Stéphane Chrétien; Alfred O. Hero

ESAIM: Probability and Statistics (2008)

  • Volume: 12, page 308-326
  • ISSN: 1292-8100

Abstract

top
In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

How to cite

top

Chrétien, Stéphane, and Hero, Alfred O.. "On EM algorithms and their proximal generalizations." ESAIM: Probability and Statistics 12 (2008): 308-326. <http://eudml.org/doc/250388>.

@article{Chrétien2008,
abstract = { In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space. },
author = {Chrétien, Stéphane, Hero, Alfred O.},
journal = {ESAIM: Probability and Statistics},
keywords = {Maximum Likelihood Estimation (MLE); EM algorithm; proximal point algorithm; Karush-Kuhn-Tucker condition; mixture densities; competing risks models; maximum likelihood estimation (MLE); expectation and maximization (EM) algorithm; proximal point algorithms},
language = {eng},
month = {5},
pages = {308-326},
publisher = {EDP Sciences},
title = {On EM algorithms and their proximal generalizations},
url = {http://eudml.org/doc/250388},
volume = {12},
year = {2008},
}

TY - JOUR
AU - Chrétien, Stéphane
AU - Hero, Alfred O.
TI - On EM algorithms and their proximal generalizations
JO - ESAIM: Probability and Statistics
DA - 2008/5//
PB - EDP Sciences
VL - 12
SP - 308
EP - 326
AB - In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.
LA - eng
KW - Maximum Likelihood Estimation (MLE); EM algorithm; proximal point algorithm; Karush-Kuhn-Tucker condition; mixture densities; competing risks models; maximum likelihood estimation (MLE); expectation and maximization (EM) algorithm; proximal point algorithms
UR - http://eudml.org/doc/250388
ER -

References

top
  1. H. Ahn, H. Moon and R.L. Kodell, Attribution of tumour lethality and estimation of the time to onset of occult tumours in the absence of cause-of-death information. J. Roy. Statist. Soc. Ser. C49 (2000) 157–169.  
  2. M.J. Box, A new method of constrained optimization and a comparison with other methods. Comp. J.8 (1965) 42–52.  
  3. G. Celeux, S. Chretien, F. Forbes and A. Mkhadri, A component-wise EM algorithm for mixtures. J. Comput. Graph. Statist.10 (2001), 697–712 and INRIA RR-3746, Aug. 1999.  
  4. S. Chretien and A.O. Hero, Acceleration of the EM algorithm via proximal point iterations, in Proceedings of the International Symposium on Information Theory, MIT, Cambridge (1998) 444.  
  5. S. Chrétien and A. Hero, Kullback proximal algorithms for maximum-likelihood estimation. IEEE Trans. Inform. Theory46 (2000) 1800–1810.  
  6. I. Csiszár, Information-type measures of divergence of probability distributions and indirect observations. Studia Sci. Math. Hung.2 (1967) 299–318.  
  7. A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc., Ser. B39 (1977) 1–38.  
  8. I.A. Ibragimov and R.Z. Has'minskii, Statistical estimation: Asymptotic theory. Springer-Verlag, New York (1981).  
  9. Journal of Statistical Planning and Inference No. 107 (2002) 1–2.  
  10. A.T. Kalai and S. Vempala, Simulated annealing for convex optimization. Math. Oper. Res.31 (2006) 253–266.  
  11. B. Martinet, Régularisation d'inéquation variationnelles par approximations successives. Revue Francaise d'Informatique et de Recherche Operationnelle3 (1970) 154–179.  
  12. G.J. McLachlan and T. Krishnan, The EM algorithm and extensions, Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley and Sons, Inc., New York (1997).  
  13. H. Moon, H. Ahn, R. Kodell and B. Pearce, A comparison of a mixture likelihood method and the EM algorithm for an estimation problme in animal carcinogenicity studies. Comput. Statist. Data Anal.31 (1999) 227–238.  
  14. A.M. Ostrowski, Solution of equations and systems of equations. Pure and Applied Mathematics, Vol. IX. Academic Press, New York-London (1966).  
  15. R.T. Rockafellar, Monotone operators and the proximal point algorithm. SIAM J. Control Optim.14 (1976) 877–898.  
  16. M. Teboulle, Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res.17 (1992) 670–690.  
  17. P. Tseng, An analysis of the EM algorithm and entropy-like proximal point methods. Math. Oper. Res.29 (2004) 27–44.  
  18. C.F.J. Wu, On the convergence properties of the EM algorithm. Ann. Stat.11 (1983) 95–103.  
  19. Z.B. Zabinsky, Stochastic adaptive search for global optimization. Nonconvex Optimization and its Applications72. Kluwer Academic Publishers, Boston, MA (2003).  
  20. W.I. Zangwill and B. Mond, Nonlinear programming: a unified approach. Prentice-Hall International Series in Management. Prentice-Hall, Inc., Englewood Cliffs, N.J. (1969).  

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.