Hit and run as a unifying device

Hans C. Andersen; Persi Diaconis

Journal de la société française de statistique (2007)

  • Volume: 148, Issue: 4, page 5-28
  • ISSN: 1962-5197

Abstract

top
We present a generalization of hit and run algorithms for Markov chain Monte Carlo problems that is ‘equivalent’ to data augmentation and auxiliary variables. These algorithms contain the Gibbs sampler and Swendsen-Wang block spin dynamics as special cases. The unification allows theorems, examples, and heuristics developed in one domain to illuminate parallel domains.

How to cite

top

Andersen, Hans C., and Diaconis, Persi. "Hit and run as a unifying device." Journal de la société française de statistique 148.4 (2007): 5-28. <http://eudml.org/doc/93471>.

@article{Andersen2007,
abstract = {We present a generalization of hit and run algorithms for Markov chain Monte Carlo problems that is ‘equivalent’ to data augmentation and auxiliary variables. These algorithms contain the Gibbs sampler and Swendsen-Wang block spin dynamics as special cases. The unification allows theorems, examples, and heuristics developed in one domain to illuminate parallel domains.},
author = {Andersen, Hans C., Diaconis, Persi},
journal = {Journal de la société française de statistique},
keywords = {Markov chain Monte Carlo algorithms; hit and run; data augmentation; auxiliary variables; Swedsen-Wang algorithm; Burnside process},
language = {eng},
number = {4},
pages = {5-28},
publisher = {Société française de statistique},
title = {Hit and run as a unifying device},
url = {http://eudml.org/doc/93471},
volume = {148},
year = {2007},
}

TY - JOUR
AU - Andersen, Hans C.
AU - Diaconis, Persi
TI - Hit and run as a unifying device
JO - Journal de la société française de statistique
PY - 2007
PB - Société française de statistique
VL - 148
IS - 4
SP - 5
EP - 28
AB - We present a generalization of hit and run algorithms for Markov chain Monte Carlo problems that is ‘equivalent’ to data augmentation and auxiliary variables. These algorithms contain the Gibbs sampler and Swendsen-Wang block spin dynamics as special cases. The unification allows theorems, examples, and heuristics developed in one domain to illuminate parallel domains.
LA - eng
KW - Markov chain Monte Carlo algorithms; hit and run; data augmentation; auxiliary variables; Swedsen-Wang algorithm; Burnside process
UR - http://eudml.org/doc/93471
ER -

References

top
  1. [1] Aldous D., and Fill J. (2007). “Reversible Markov chains and random walks on graphs”, monograph in preparation, drafts available online. 
  2. [2] Andersen H. C. (1980). “Molecular dynamics simulations at constant pressure and/or temperature”, J. Chem. Phys. 72, 2384-2393 (1980). 
  3. [3] Belisle C., Boneh A., and Caron R. (1998). “Convergence properties of hit and run samplers”, Comm. Statist-Stochastic Models 14, 767-800. Zbl0912.65121MR1631518
  4. [4] Belisle C., Romeijn H., and Smith R. (1993). “Hit and run algorithms for generating multivariate distributions”, Math. Oper. Res. 18, 255-266. Zbl0771.60052MR1250117
  5. [5] Berti P., Platelli L., and Rigo P. (2007). “Trivial intersection of σ -fields and Gibbs sampling”, to appear Annals of Probability. Zbl1159.60007MR2308591
  6. [6] Besag J., and Green P. (1993). “Spatial statistics and Bayesian computation” (with discussion), J. Roy. Statist. Soc. B 16, 395-407. Zbl0800.62572MR1210422
  7. [7] Blackwell D., and Ryll-Nardzewski C. (1963). “Non-existence of everywhere proper conditional distributions”, Ann. Math. Statist. 34, 223-225. Zbl0122.13202MR148097
  8. [8] Blanchet J., and Meng X. (2007). “Exact sampling, regeneration and minorization conditions”, preprint, Dept. of Statistics, Harvard University. 
  9. [9] Boneh A., and Golan A. (1979). “Constraints redundancy and feasible region boundedness by random feasible point generator (RGPG)”, Third European Congress on Operations Research – EURO III, Amsterdam. 
  10. [10] Borgs C., Chayes J., Frieze A., Kim J., Tetali P., Vigoda E., and Vu V. (1999). “Torpid mixing of some MCMC algorithms in statistical physics”, Proc. 40th IEEE Symp. on Foundations of Computer Science (FOCS) 218-229. MR1917562
  11. [11] Borgs C., Chase J., Dyer M., and Tetali P. (2004). “On the sampling problem for H-coloring on the hypercubic lattice”, DIMACS Series in Discrete Math. and Computer Science 63, 13-28. Zbl1074.05032MR2056236
  12. [12] Borovkov K. (1991). “A New Variant of the Monte Carlo Method”, Th. Probabl. Appl. 36, 355-360. Zbl0776.65067MR1119508
  13. [13] Breiman L. (1968). “Probability”, Addison-Wesley, Reading, Mass. Zbl0174.48801MR229267
  14. [14] Ceperley D. M. (1995). “Path integrals in the theory of condensed helium”, Rev. Mod. Phys. 67, 279-355. 
  15. [15] Chen Y., Diaconis P., Holmes S., and Liu J. (2005). “Sequential Monte-Carlo methods for statistical analysis of tables”, Jour. Amer. Statist. Assoc. 100, 109-120. Zbl1117.62310MR2156822
  16. [16] Chen M., Shao Q., and Ibrahim J. (2000). “Monte Carlo Methods in Bayesian Computation”, Springer, New York. Zbl0949.65005MR1742311
  17. [17] Chen M., and Schmeiser B. (1993). “Performance of Gibbs, hit and run and Metropolis samplers”, Jour. Comput. Graph. Statist. 2, 251-272. MR1272394
  18. [18] Comets F., Popov S., Schutz G., and Vachkovskaia V. (2007). “Billiards in a general domain with random reflectors”, arXiv:math 061279941. Zbl1173.37319
  19. [19] Consta S., Wilding N. B., Frenkel D., and Alexandrowicz Z. (1999). “Recoil growth: An efficient simulation method for multi-polymer systems”, J. Chem. Phys. 110, 3220-3228. 
  20. [20] Cryan M., Dyer M., Goldberg L., and Jerrum M. (2006). “Rapidly mixing Markov chains for sampling contingency tables with a constant number of rows”, SIAM J. Comput. 36, 247-278. Zbl1117.62062MR2231648
  21. [21] Damien P., Walker S., and Wakefield J. (1999). “Gibbs sampling for Bayesian nonconjugate models using auxiliary variables”, J. Roy. Statist. Soc. B 61, 331-344. Zbl0913.62028MR1680334
  22. [22] Diaconis P. (1988). “Group representations in probability and statistics”, IMS, Hayward, Mass. Zbl0695.60012MR964069
  23. [23] Diaconis P. (2003). “Analysis of a Bose-Einstein Markov chain”, Annal. Institut H. Poincaré PR 41, 409-418. Zbl1130.60012MR2139027
  24. [24] Diaconis P., and Efron B. (1985). “Testing for independendence in a two-way table: New interpretations of the chi-square statistic”, Ann. Statist. 13, 845-913. Zbl0593.62040MR803747
  25. [25] Diaconis P., and Gangolli A. (1994). “Rectangular arrays with fixed margins”, in Discrete Probability and Algorithms, D. Aldous et al., eds., Springer-Verlag, New York. Zbl0839.05005MR1380519
  26. [26] Diaconis P., Graham R., and Holmes S. (1999). “Statistical problems involving permutations with restricted positions”, in M. de Gunst et al., ed., “State of the art in probability and statistics”, IMS, Benchwood, OH., pp. 195-222. MR1836562
  27. [27] Diaconis P., and Holmes S. (2004). “Stein’s Method: Expository Lectures and Applications”, Institute of Mathematical Statistics, Beachwood, Ohio; Chapter 1. Zbl1079.62024MR2118599
  28. [28] Diaconis P., Holmes S., and Neal R. (2000). “Analysis of a non-reversible Markov chain sampler”, Annals Appl. Probab. 10, 726-752. Zbl1083.60516MR1789978
  29. [29] Diaconis P., Khare K., and Saloff-Coste L. (2007). “Gibbs sampling, exponential families and orthogonal polynomials”, to appear Statistical Science. Zbl1327.62058MR2446500
  30. [30] Diaconis P., Khare K., and Saloff-Coste L. (2007A). “Gibbs sampling, exponential families and coupling”, preprint, Dept. of Statistics, Stanford University. Zbl1327.62058
  31. [31] Diaconis P., Khare K., and Saloff-Coste L. (2007B). “Stochastic alternating projections”, preprint, Dept. of Statistics, Stanford University. Zbl1268.60098
  32. [32] Diaconis P., and Ram A. (2000). “Analysis of systematic scan Metropolis algorithms using Iwahori-Hecke algebra techniques”, Michigan Jour. Math. 48, 157-190. Zbl0998.60069MR1786485
  33. [33] Diaconis P., and Sturmfels B. (1998). “Algebraic algorithms for sampling from conditional distributions”, Annals Statist. 26, 363-397. Zbl0952.62088MR1608156
  34. [34] Duane S., Kennedy A. D., Pendleton B. J., and Roweth D. (1987). “Hybrid Monte Carlo”, Phys. Lett. B 195, 216-222. 
  35. [35] Dyer M., Goldberg L., and Jerrum M. (2006). “Systematic scan for sampling colorings”, Ann. Appl. Probab. 16, 185-230. Zbl1095.60024MR2209340
  36. [36] Edwards R. O., and Sokal A. D. (1988). “Generalization of the Fortuin-Kasteleyn-Swendsen-Wang representation and Monte Carlo Algorithm”, Phys. Rev. D 38, 2009-2012. MR965465
  37. [37] Frenkel D., Mooij G. C. A. M., and Smit B. (1991). “Novel scheme to study structural and thermal properties of continuously deformable molecules”, J. Phys.: Condens. Matter 3, 3053-3076. 
  38. [38] Frenkel D. and Smit B. (2001). “Understanding Molecular Simulation - From Algorithms to Applications”, 2nd edition, Academic Press, San Diego. Zbl0889.65132
  39. [39] Galvin D., and Tetali P. (2004). “Slow mixing of the Glauber dynamics for the hard core model on the Hamming cube”, Proc. Annual Symp. Disc. Algo. (SODA) 459-460. 
  40. [40] Geyer C. J. (1991). “Markov chain Monte Carlo maximum likelihood”, in E. Keramigas (ed.) “Computing Science and Statistics: The 23rd symposium on the interface”, Interface Foundation, Fairfax, pp. 156-163. 
  41. [41] Goldberg L., and Jerrum M. (2002). “The Burnside process mixes slowly”, Combinatorics, Probability, and Computing 11, 21-34. Zbl1008.68085MR1888180
  42. [42] Gore V., and Jerrum M. (1999). “The Swendsen Wang process does not always mix rapidly”, J. Stat. Phys. 97, 67-86. Zbl1006.82015MR1733467
  43. [43] Higdon D. (1998). “Auxiliary variable methods for Markov chain Monte Carlo with applications”, Jour. Amer. Statist. Assoc. No. 442, 585-595. Zbl0953.62103
  44. [44] Hukushima K., and Nemoto K. (1996). “Exchange Monte Carlo method and application to spin glass simulations”, J. Phys. Soc. Japan 65, 1604-1608. 
  45. [45] Jerrum M. (1993). “Uniform sampling modulo a group of symmetries using Markov chain simulation”, DIMACS Series in Discrete Math 10, 37-47. Zbl0814.68077MR1235566
  46. [46] Kiatsupaibul S., Smith R., and Zabinsky Z. (2002). “A discrete hit and run algorithm for generating samples from general discrete multivariate distributions”, preprint, Dept. of Industrial Relations and Operations Engineering, University of Michigan 
  47. [47] Kong A., Meng X., McCullagh P., Nicolae D., and Tan Z. (2003). “A theory of statistical models for Monte-Carlo integration” (with discussion), J. Roy. Statist. Soc. B 65, 585-618. Zbl1067.62054MR1998624
  48. [48] Liu J. S. (2001). “Monte Carlo Strategies in Scientific Computing”, Springer-Verlag, New York. Zbl0991.65001MR1842342
  49. [49] Lovasz L. (1999). “Hit and run mixes fast”, Math. Program. 86, 443-461. Zbl0946.90116MR1733749
  50. [50] Lovasz L., and Vempala S. (2003). “Hit and run is fast and fun”, preprint, Microsoft Research. 
  51. [51] Lovasz L., and Vempala S. (2006). “Hit and run from a corner”, SIAM J. Comput. 35, 985-1005. Zbl1103.52002MR2203735
  52. [52] Mallows C. (1957). “Non-null ranking models, I.” Biometrika 44, 114-130. Zbl0087.34001MR87267
  53. [53] Marden J. (1995). “Analyzing and modeling rank data”, Chapman and Hall, London. Zbl0853.62006MR1346107
  54. [54] Neal R. (2003). “Slice sampling”, Annals of Statist. 31, 705-767. Zbl1051.65007MR1994729
  55. [55] Pak I. (2001). “What do we know about the product replacement algorithm?”, in “Groups and Computation III”, Kantor, W., and Seress, A. eds., De Gruyter, Berlin, pp. 301-347. Zbl0986.68172MR1829489
  56. [56] Roberts G. and Rosenthal J. (1999). “Convergence of slice sampler Markov chains”, J. Royal Statist. Soc. B 61, 643-660. Zbl0929.62098MR1707866
  57. [57] Smith R. (1984). “Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions”, Oper. Res. 32, 1296-1308. Zbl0552.65004MR775260
  58. [58] Swensen R. H., and Wang J.-S. (1987). “Nonuniversal critical dynamics in Monte Carlo simulations”, Phys. Rev. Lett. 58, 86-88. 
  59. [59] Tanner M., and Wong W. (1987). “The calculation of posterior distributions by data augmentation”, J. Amer. Statist. Assoc. 82, 528-550. Zbl0619.62029MR898357
  60. [60] Ter Braak C. J. F. (2006). “A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces”, Stat Comput 16, 239-249. MR2242236
  61. [61] Turcin V. (1971). “On the computation of multidimensional integrals by the Monte Carlo Method”, Th. Probabl. Appl. 16, 720-724. Zbl0257.65030MR292259
  62. [62] Van der Vaart A. (2002). “The statistical work of Lucian Le Cam”, Ann. Statist. 30, 631-682. Zbl1103.62301MR1922537
  63. [63] Van Dyk D., and Meng X. (2001). “The art of data augmentation”, Jour. Comp. Graph. Statist. 10, 1-111. MR1936358
  64. [64] Yang R., and Berger J. (1994). “Estimation of a covariance matrix using the reference prior”, Ann. Statist. 22, 1195-1211. Zbl0819.62013MR1311972

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.