Investigations particulaires pour l’inférence statistique et l’optimisation de plan d’expériences

Éric Parent; Billy Amzal; Philippe Girard

Journal de la société française de statistique (2008)

  • Volume: 149, Issue: 1, page 27-51
  • ISSN: 1962-5197

Abstract

top
Particle algorithms are Monte Carlo techniques that put together steps of importance sampling, bootstrap resampling, markovian rejuvenating and simulated annealing. We develop three examples of increasing complexity and explain how to implement such algorithms for maximum likelihood search, for inference of a model with latent variables and for optimal design. Since we believe that particle algorithms will soon become tools of choice for statistical practitioners, their results are compared with the known solutions of these rather common examples so as to test the algorithms’ performances and to show their limits.

How to cite

top

Parent, Éric, Amzal, Billy, and Girard, Philippe. "Investigations particulaires pour l’inférence statistique et l’optimisation de plan d’expériences." Journal de la société française de statistique 149.1 (2008): 27-51. <http://eudml.org/doc/93473>.

@article{Parent2008,
abstract = {Les algorithmes particulaires sont des techniques de Monte-Carlo qui associent des étapes d’échantillonnage pondéré, de rééchantillonnage bootstrap, de régénérescence markovienne et de recuit simulé. Grâce à trois exemples de complexité croissante, nous décrivons leurs implémentations pour l’estimation du maximum de vraisemblance, l’évaluation de la distribution a posteriori pour un modèle à variables latentes et la recherche du plan d’expérience optimal. Les solutions de ces exemples pédagogiques illustrent les performances et les limites de ces algorithmes, promis à une place de choix dans la trousse à outils du statisticien.},
author = {Parent, Éric, Amzal, Billy, Girard, Philippe},
journal = {Journal de la société française de statistique},
keywords = {particle algorithms; Monte Carlo simulation; optimal experimental design; bayesian inference},
language = {fre},
number = {1},
pages = {27-51},
publisher = {Société française de statistique},
title = {Investigations particulaires pour l’inférence statistique et l’optimisation de plan d’expériences},
url = {http://eudml.org/doc/93473},
volume = {149},
year = {2008},
}

TY - JOUR
AU - Parent, Éric
AU - Amzal, Billy
AU - Girard, Philippe
TI - Investigations particulaires pour l’inférence statistique et l’optimisation de plan d’expériences
JO - Journal de la société française de statistique
PY - 2008
PB - Société française de statistique
VL - 149
IS - 1
SP - 27
EP - 51
AB - Les algorithmes particulaires sont des techniques de Monte-Carlo qui associent des étapes d’échantillonnage pondéré, de rééchantillonnage bootstrap, de régénérescence markovienne et de recuit simulé. Grâce à trois exemples de complexité croissante, nous décrivons leurs implémentations pour l’estimation du maximum de vraisemblance, l’évaluation de la distribution a posteriori pour un modèle à variables latentes et la recherche du plan d’expérience optimal. Les solutions de ces exemples pédagogiques illustrent les performances et les limites de ces algorithmes, promis à une place de choix dans la trousse à outils du statisticien.
LA - fre
KW - particle algorithms; Monte Carlo simulation; optimal experimental design; bayesian inference
UR - http://eudml.org/doc/93473
ER -

References

top
  1. [1] Amzal B., Bois F., Parent E., Robert C.P.2006. Bayesian Optimal Design Via Interacting Particle Systems. Journal of the American Statistical Association, 101(474), 773–785. Zbl1119.62308MR2281248
  2. [2] Andrieu C., Doucet A.2000. Simulated Annealing for Maximum a posteriori Parameter Estimation of Hidden Markov Models. IEEE Trans. Information Theory, 46(3), 994–1004. Zbl1009.62070MR1763472
  3. [3] Andrieu C., Doucet A., Singh S., Tadic V.2004. Particle Methods for Change Detection, System Identification, and Control. Proceedings of the IEEE, 92(3), 423–438. 
  4. [4] Arulampalam M. S., Maskell S., Gordon N., Clapp T.2002. A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking. IEEE Transaction on Signal Processing, 50(2), 174–188. 
  5. [5] Barnett V.1973. Bayesian and Decision Theoric Methods Applied to Industrial Problems. The statistician, 22(3), 199–226. 
  6. [6] Berger J. O.1985. Statistical Decision Theory and Bayesian Analysis. Springer Verlag, New York. Zbl0572.62008MR804611
  7. [7] Bernier J., Parent E., Boreux J-J.2000. Statistique de l’Environnement. Traitement Bayésien de l’Incertitude. Paris : Lavoisier. 
  8. [8] Brooks S P. 1998. Markov Chain Monte Carlo Method and its Application. The Statistician, 47, 69–100. 
  9. [9] Brooks S.P.2003. Bayesian Computation : A Statistical Revolution. Trans. Roy. Statist. Soc., series A, 15, 2681–2697. Zbl1044.62029MR2050672
  10. [10] Brooks S.P., Morgan B.J.T.1995. Optimization Using Simulated Annealing. The Statistician, 44, 241–257. 
  11. [11] Cappé O., Guilin A., Marin J.M., Robert C.P.2004. Population Monte Carlo. Journal of Computational and Graphical Statistics, 13(4), 907–929. MR2109057
  12. [12] Cappé O., Moulines E., Rydèn T.2005. Inference in Hidden Markov Models. Springer. Zbl1080.62065MR2159833
  13. [13] Carlin B.P., Kadane J.B., Gelfand A.E.1998. Approaches for Optimal Sequential Decision Analysis in Clinical Trials. Biometrics, 54(3), 964–975. Zbl1058.62548
  14. [14] Chopin N.2002. A Sequential Particle Filter Method for Static Models. Biometrika, 89, 539–552. Zbl1036.62062MR1929161
  15. [15] Chopin N.2004. Central Limit Theorem for Sequential Monte Carlo Methods and its Application to Bayesian Inference. Ann. Stat., 32(6), 2385–2411. Zbl1079.65006MR2153989
  16. [16] Del Moral P.2004. Feynman-Kac Formulae : Genealogical and Interacting Particle Systems with Applications. Springer-Verlag. Zbl1130.60003MR2044973
  17. [17] Del Moral P., Guionnet A.1999. Central Limit Theorem for Nonlinear Filtering and Interacting Particule Systems. Ann. Appl. Prob., 155–194. Zbl0938.60022MR1687359
  18. [18] Del Moral P., Kallel L., Rowe J.2001. Natural Computing Series : Theoretical Aspects of Evolutionary Computing. Springer-Verlag, Berlin. Chap. Modelling Genetic Algorithms with Interacting Particle Systems, pages 10–67. 
  19. [19] Douc R., Cappé O.2005. Comparison of Resampling Schemes for Particle Filtering. Pages 64–69 of : Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis. 
  20. [20] Douc R., Cappé O., Moulines E., Robert C.P.2002. On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models. Scandinavian Journal of Statistics, 29(4), 615–635. Zbl1035.62015MR1988415
  21. [21] Douc R., Guilin A., Marin J.M., Robert C.P.2007. Convergence of Adaptive Mixtures of Importance Sampling Schemes. Ann. Stat., 35(1), 420–448. Zbl1132.60022MR2332281
  22. [22] Doucet A., de Freitas N., Gordon N.2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag. Zbl0967.00022MR1847783
  23. [23] Doucet A., Godsill J.M., Robert C.P.2002. Population Marginal maximum a posteriori Estimation Using Markov Chain Monte Carlo. Statistics and Computing, 12, 77–84. Zbl1247.62087MR1877581
  24. [24] Doucet A., Del Moral P., Peters G.W.2004. Sequential Monte Carlo Samplers. Tech. rept. Cambridge University. Zbl1105.62034
  25. [25] Gelman A., Rubin D. B.1992. Inference from Iterative Simulation Using Multiple Sequences. Statistical Science, 457–472. 
  26. [26] Gelman A., Carlin J. B., Stern H. S., Rubin D. B.1995. Bayesian Data Analysis. London : Chapman and Hall. Zbl0914.62018MR1385925
  27. [27] Geman S., Geman D.1984. Stochastic Relaxation, Gibbs Distribution and the Bayesian Restoration of Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–741. Zbl0573.62030
  28. [28] Gilks W.R., Richardson S., Spiegelhalter D.1996. Markov Chain Monte Carlo in Practice. London : Chapman and Hall. Zbl0832.00018MR1397966
  29. [29] Kass R. E., Carlin B. P., Gelman A., Neal R. M.1998. Markov Chain Monte Carlo in Practice : A Roundtable Discussion. The American Statistician, 52(2), 93–100. MR1628427
  30. [30] Kuhn E., Lavielle M.2004. Coupling a Stochastic Approximation Version of EM with an MCMC Procedure. ESAIM, Probab. Stat., 8, 115–131. Zbl1155.62420MR2085610
  31. [31] Künsch H.2001. State Space and Hidden Markov Models. Pages 109–173 of : Barndor-Nielsen, O. E., Cox, D. R., Klüppelberg, C. (eds), Complex Stochastic Systems. Chapman and Hall. Zbl1002.62072MR1893412
  32. [32] Künsch H.2005. Recursive Monte Carlo Filters. Ann. Stat., 1983–2021. Zbl1086.62106
  33. [33] Liu J.S., Chen R.1998. Sequential Monte Carlo Methods for Dynamic Systems. Journal of American Statistical Association, 93(443), 113–119. Zbl1064.65500MR1649198
  34. [34] Mc Cullagh P., Nelder J.A.1989. Generalized Linear Models. second edn. Chapman and Hall/CRC. 
  35. [35] Mengersen K.L., Robert C.P., Guihennec-Jouyaux C.1999. MCMC Convergence Diagnostics : A Reviewww (with Discussion). Pages 415–440 of : Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds), Bayesian Stat., vol. 6. Oxford University Press. Zbl0957.62019MR1723507
  36. [36] Metropolis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.H, Teller E.1953. Equations of State Calculations by Fast Computing Machines. J. Chem. Phys., 21, 1087–1091. 
  37. [37] Müller P.1999. Simulation-Based Optimal Design. Pages 459–474 of : Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds), Bayesian Stat., vol. 6. Oxford University Press. Zbl0974.62058MR1723509
  38. [38] Munier B., Parent E.1998. Le Développement Récent des Sciences de la Décision : Un Regard Critique sur la Statistique Décisionnelle Bayésienne. In : Parent, E., Hubert, P., Bobée, B., Miquel, J. (eds), Bayesian Methods in Hydrol. Sci. UNESCO Publishing. 
  39. [39] Oujdane N., Ruberthaler S.2005. Stability and Uniform Particle Approximation of Nonlinear Filters in case of non Ergodic Signals. Stochastic Analysis and Applications, 23, 421–448. Zbl1140.93485MR2140972
  40. [40] Parent E., Bernier J.2007. Le Raisonnement Bayésien : Modélisation et Inférence. Paris : Springer Verlag France. Zbl1284.62017
  41. [41] Parent, E., Chaouche A., Girard P.1995. Sur l’Apport des Statistiques Bayésiennes au Contrôle de la Qualité par Attribut, partie 1 : Contrôle Simple. Rev. Statistique Appliquée, XLIII(4), 5–18. 
  42. [42] Parent E., Lang G., Girard P.1996. Sur l’Apport des Statistiques Bayésiennes au Contrôle de la Qualité par Attribut, partie 2 : Contrôle Séquentiel Tronqué. Rev. Statistique Appliquée, XLIV(1), 37–54. 
  43. [43] Pitt M., Shephard N.1999. Filtering Via Simulation : Auxiliary Particle Filters. Ecological Applications, 94(446), 590–599. Zbl1072.62639MR1702328
  44. [44] Reeves C.R., Wright C.C.1996. Genetic Algorithms and the Design of Experiments. In : Proc. IMA Fall Workshop on Evolutionary Algorithms. 
  45. [45] Robert C. P.2006. Le Choix Bayésien : Principes et Pratique. Paris : Springer Verlag France. 
  46. [46] Robert C.P., Casella G.2004. Monte-Carlo Statistical Methods. Springer-Verlag. Zbl1096.62003MR2080278
  47. [47] Rubin D.B.1988. Using the SIR Algorithm to Simulate Posterior Distributions. Pages 395–402 of : Bayesian Statistics 3, vol. 3. Oxford University Press. Zbl0713.62035
  48. [48] Smith A.F.M., Gelfand A.E.1992. Bayesian Statistics without Tears : a Sampling-Resampling Perspective. The American Statistician, 46, 84–88. MR1165566
  49. [49] Tagaras G.1986. Economic Design of Acceptance Sampling and Process Control Procedures for Quality Assurance in Complex Production Systems. Ph.D. thesis, Stanford University. 
  50. [50] Tanner M. H.1992. Tools for Statistical Inference : Observed Data and Data Augmentation Methods. New York : Springer-Verlag. Zbl0724.62003MR1096956
  51. [51] Van Laarhoven P.J.M., Aarts E.H.L.1987. Simulated Annealing : Theory and Applications. Reider Pub. and Kluwer Dordrecht, Holland. Zbl0643.65028MR904050
  52. [52] Von Neumann J., Morgenstern O.1944. Theory of Games and Economic Behavior. Princeton Univ. Press, Princeton NJ. Zbl1109.91001MR11937

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.