Coupling a stochastic approximation version of EM with an MCMC procedure
ESAIM: Probability and Statistics (2010)
- Volume: 8, page 115-131
- ISSN: 1292-8100
Access Full Article
top
Abstract
topHow to cite
topKuhn, Estelle, and Lavielle, Marc. "Coupling a stochastic approximation version of EM with an MCMC procedure." ESAIM: Probability and Statistics 8 (2010): 115-131. <http://eudml.org/doc/104313>.
@article{Kuhn2010,
abstract = {
The stochastic approximation version of EM (SAEM) proposed by Delyon et al. (1999) is
a powerful alternative to EM when the E-step is intractable. Convergence of
SAEM toward a maximum of the observed likelihood is established when
the unobserved data are simulated at each iteration under the conditional
distribution. We show that this very restrictive assumption can be weakened. Indeed,
the results of Benveniste et al. for stochastic approximation
with Markovian perturbations are used to establish the convergence
of SAEM when it is coupled with a Markov chain Monte-Carlo
procedure. This result is very useful for many practical applications.
Applications to the convolution model and the change-points model are presented to illustrate the proposed method.
},
author = {Kuhn, Estelle, Lavielle, Marc},
journal = {ESAIM: Probability and Statistics},
keywords = {EM algorithm; SAEM algorithm; stochastic
approximation; MCMC algorithm; convolution model; change-points model.; stochastic approximation; change-points model},
language = {eng},
month = {3},
pages = {115-131},
publisher = {EDP Sciences},
title = {Coupling a stochastic approximation version of EM with an MCMC procedure},
url = {http://eudml.org/doc/104313},
volume = {8},
year = {2010},
}
TY - JOUR
AU - Kuhn, Estelle
AU - Lavielle, Marc
TI - Coupling a stochastic approximation version of EM with an MCMC procedure
JO - ESAIM: Probability and Statistics
DA - 2010/3//
PB - EDP Sciences
VL - 8
SP - 115
EP - 131
AB -
The stochastic approximation version of EM (SAEM) proposed by Delyon et al. (1999) is
a powerful alternative to EM when the E-step is intractable. Convergence of
SAEM toward a maximum of the observed likelihood is established when
the unobserved data are simulated at each iteration under the conditional
distribution. We show that this very restrictive assumption can be weakened. Indeed,
the results of Benveniste et al. for stochastic approximation
with Markovian perturbations are used to establish the convergence
of SAEM when it is coupled with a Markov chain Monte-Carlo
procedure. This result is very useful for many practical applications.
Applications to the convolution model and the change-points model are presented to illustrate the proposed method.
LA - eng
KW - EM algorithm; SAEM algorithm; stochastic
approximation; MCMC algorithm; convolution model; change-points model.; stochastic approximation; change-points model
UR - http://eudml.org/doc/104313
ER -
References
top- A. Benveniste, M. Métivier and P. Priouret, Adaptive algorithms and stochastic approximations. Springer-Verlag, Berlin (1990). Translated from the French by Stephen S. Wilson.
- O. Brandière and M. Duflo, Les algorithmes stochastiques contournent-ils les pièges ? C. R. Acad. Sci. Paris Ser. I Math.321 (1995) 335–338.
- H.F. Chen, G. Lei and A.J. Gao, Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds. Stochastic Process. Appl.27 (1988) 217–231.
- D. Concordet and O.G. Nunez, A simulated pseudo-maximum likelihood estimator for nonlinear mixed models. Comput. Statist. Data Anal.39 (2002) 187–201.
- B. Delyon, M. Lavielle and E. Moulines, Convergence of a stochastic approximation version of the EM algorithm. Ann. Statist.27 (1999) 94–128.
- A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B39 (1977) 1–38.
- M.G. Gu and F.H. Kong, A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems. Proc. Natl. Acad. Sci. USA95 (1998) 7270–7274 (electronic).
- M.G. Gu and H.-T. Zhu, Maximum likelihood estimation for spatial models by Markov chain Monte Carlo stochastic approximation. J. R. Stat. Soc. Ser. B63 (2001) 339–355.
- K. Lange, A gradient algorithm locally equivalent to the EM algorithm. J. R. Stat. Soc. Ser. B57 (1995) 425–437.
- M. Lavielle and E. Lebarbier, An application of MCMC methods to the multiple change-points problem. Signal Processing81 (2001) 39–53.
- M. Lavielle and E. Moulines, A simulated annealing version of the EM algorithm for non-Gaussian deconvolution. Statist. Comput.7 (1997) 229–236.
- X.-L. Meng and D.B. Rubin, Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika80 (1993) 267–278.
- K.L. Mengersen and R.L. Tweedie, Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist.24 (1996) 101–121.
- S.P. Meyn and R.L. Tweedie, Markov chains and stochastic stability, Springer-Verlag London Ltd., London. Comm. Control Engrg. Ser. (1993).
- C.-F. Jeff Wu, On the convergence properties of the EM algorithm. Ann. Statist.11 (1983) 95–103.
- J.-F. Yao, On recursive estimation in incomplete data models. Statistics34 (2000) 27–51 (English).
NotesEmbed ?
topYou must be logged in to post comments.
To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.