On fine properties of mixtures with respect to concentration of measure and Sobolev type inequalities

Djalil Chafaï; Florent Malrieu

Annales de l'I.H.P. Probabilités et statistiques (2010)

  • Volume: 46, Issue: 1, page 72-96
  • ISSN: 0246-0203

Abstract

top
Mixtures are convex combinations of laws. Despite this simple definition, a mixture can be far more subtle than its mixed components. For instance, mixing gaussian laws may produce a potential with multiple deep wells. We study in the present work fine properties of mixtures with respect to concentration of measure and Sobolev type functional inequalities. We provide sharp Laplace bounds for Lipschitz functions in the case of generic mixtures, involving a transportation cost diameter of the mixed family. Additionally, our analysis of Sobolev type inequalities for two-component mixtures reveals natural relations with some kind of band isoperimetry and support constrained interpolation via mass transportation. We show that the Poincaré constant of a two-component mixture may remain bounded as the mixture proportion goes to 0 or 1 while the logarithmic Sobolev constant may surprisingly blow up. This counter-intuitive result is not reducible to support disconnections, and appears as a reminiscence of the variance-entropy comparison on the two-point space. As far as mixtures are concerned, the logarithmic Sobolev inequality is less stable than the Poincaré inequality and the sub-gaussian concentration for Lipschitz functions. We illustrate our results on a gallery of concrete two-component mixtures. This work leads to many open questions.

How to cite

top

Chafaï, Djalil, and Malrieu, Florent. "On fine properties of mixtures with respect to concentration of measure and Sobolev type inequalities." Annales de l'I.H.P. Probabilités et statistiques 46.1 (2010): 72-96. <http://eudml.org/doc/242922>.

@article{Chafaï2010,
abstract = {Mixtures are convex combinations of laws. Despite this simple definition, a mixture can be far more subtle than its mixed components. For instance, mixing gaussian laws may produce a potential with multiple deep wells. We study in the present work fine properties of mixtures with respect to concentration of measure and Sobolev type functional inequalities. We provide sharp Laplace bounds for Lipschitz functions in the case of generic mixtures, involving a transportation cost diameter of the mixed family. Additionally, our analysis of Sobolev type inequalities for two-component mixtures reveals natural relations with some kind of band isoperimetry and support constrained interpolation via mass transportation. We show that the Poincaré constant of a two-component mixture may remain bounded as the mixture proportion goes to 0 or 1 while the logarithmic Sobolev constant may surprisingly blow up. This counter-intuitive result is not reducible to support disconnections, and appears as a reminiscence of the variance-entropy comparison on the two-point space. As far as mixtures are concerned, the logarithmic Sobolev inequality is less stable than the Poincaré inequality and the sub-gaussian concentration for Lipschitz functions. We illustrate our results on a gallery of concrete two-component mixtures. This work leads to many open questions.},
author = {Chafaï, Djalil, Malrieu, Florent},
journal = {Annales de l'I.H.P. Probabilités et statistiques},
keywords = {transportation cost distances; Mallows or Wasserstein distance; mixtures of distributions; finite gaussian mixtures; concentration of measure; gaussian bounds; tails probabilities; deviation inequalities; functional inequalities; Poincaré inequalities; Gross logarithmic Sobolev inequalities; band isoperimetry; transportation of measure; mass transportation; finite Gaussian mixtures; Gaussian bounds},
language = {eng},
number = {1},
pages = {72-96},
publisher = {Gauthier-Villars},
title = {On fine properties of mixtures with respect to concentration of measure and Sobolev type inequalities},
url = {http://eudml.org/doc/242922},
volume = {46},
year = {2010},
}

TY - JOUR
AU - Chafaï, Djalil
AU - Malrieu, Florent
TI - On fine properties of mixtures with respect to concentration of measure and Sobolev type inequalities
JO - Annales de l'I.H.P. Probabilités et statistiques
PY - 2010
PB - Gauthier-Villars
VL - 46
IS - 1
SP - 72
EP - 96
AB - Mixtures are convex combinations of laws. Despite this simple definition, a mixture can be far more subtle than its mixed components. For instance, mixing gaussian laws may produce a potential with multiple deep wells. We study in the present work fine properties of mixtures with respect to concentration of measure and Sobolev type functional inequalities. We provide sharp Laplace bounds for Lipschitz functions in the case of generic mixtures, involving a transportation cost diameter of the mixed family. Additionally, our analysis of Sobolev type inequalities for two-component mixtures reveals natural relations with some kind of band isoperimetry and support constrained interpolation via mass transportation. We show that the Poincaré constant of a two-component mixture may remain bounded as the mixture proportion goes to 0 or 1 while the logarithmic Sobolev constant may surprisingly blow up. This counter-intuitive result is not reducible to support disconnections, and appears as a reminiscence of the variance-entropy comparison on the two-point space. As far as mixtures are concerned, the logarithmic Sobolev inequality is less stable than the Poincaré inequality and the sub-gaussian concentration for Lipschitz functions. We illustrate our results on a gallery of concrete two-component mixtures. This work leads to many open questions.
LA - eng
KW - transportation cost distances; Mallows or Wasserstein distance; mixtures of distributions; finite gaussian mixtures; concentration of measure; gaussian bounds; tails probabilities; deviation inequalities; functional inequalities; Poincaré inequalities; Gross logarithmic Sobolev inequalities; band isoperimetry; transportation of measure; mass transportation; finite Gaussian mixtures; Gaussian bounds
UR - http://eudml.org/doc/242922
ER -

References

top
  1. [1] C. Ané, S. Blachère, D. Chafaï, P. Fougères, I. Gentil, F. Malrieu, C. Roberto and G. Scheffer. Sur les inégalités de Sobolev logarithmiques. Société Mathématique de France, Paris, 2000. (Preface by D. Bakry and M. Ledoux.) Zbl0982.46026MR1845806
  2. [2] D. Bakry and M. Émery. Diffusions hypercontractives. In Séminaire de probabilités (XIX, 1983/84) 177–206. Lecture Notes in Math. 1123. Springer, Berlin, 1985. Zbl0561.60080MR889476
  3. [3] D. Bakry, M. Ledoux and F.-Y. Wang. Perturbations of functional inequalities using growth conditions. J. Math. Pures Appl. (9) 87 (2007) 394–407. Zbl1120.60070MR2317340
  4. [4] F. Barthe, P. Cattiaux and C. Roberto. Interpolated inequalities between exponential and Gaussian, Orlicz hypercontractivity and isoperimetry. Rev. Mat. Iberoamericana 22 (2006) 993–1067. Zbl1118.26014MR2320410
  5. [5] F. Barthe and C. Roberto. Sobolev inequalities for probability measures on the real line. Studia Math. 159 (2003) 481–497. (Dedicated to Professor Aleksander Pełczyński on the occasion of his 70th birthday (Polish).) Zbl1072.60008MR2052235
  6. [6] S. G. Bobkov. Concentration of normalized sums and a central limit theorem for noncorrelated random variables. Ann. Probab. 32 (2004) 2884–2907. Zbl1065.60006MR2094433
  7. [7] S. G. Bobkov. Generalized symmetric polynomials and an approximate de Finetti representation. J. Theoret. Probab. 18 (2005) 399–412. Zbl1079.28001MR2137450
  8. [8] S. G. Bobkov and F. Götze. Exponential integrability and transportation cost related to logarithmic Sobolev inequalities. J. Funct. Anal. 163 (1999) 1–28. Zbl0924.46027MR1682772
  9. [9] F. Bolley and C. Villani. Weighted Csiszár–Kullback–Pinsker inequalities and applications to transportation inequalities. Ann. Fac. Sci. Toulouse Math. 14 (2005) 331–352. Zbl1087.60008MR2172583
  10. [10] L. A. Caffarelli. Monotonicity properties of optimal transportation and the FKG and related inequalities. Comm. Math. Phys. 214 (2000) 547–563. Zbl0978.60107MR1800860
  11. [11] L. A. Caffarelli. Erratum: [Comm. Math. Phys. 214 (2000) 547–563]. Comm. Math. Phys. 225 (2002) 449–450. Zbl0978.60107
  12. [12] E. A. Carlen. Superadditivity of Fisher’s information and logarithmic Sobolev inequalities. J. Funct. Anal. 101 (1991) 194–211. Zbl0732.60020MR1132315
  13. [13] S. Chatterjee. Spin glasses and Stein’s method. Preprint, 2007. Available at arXiv:0706.3500v2 [math.PR]. Zbl1209.82016
  14. [14] P. Diaconis and L. Saloff-Coste. Logarithmic Sobolev inequalities for finite Markov chains. Ann. Appl. Probab. 6 (1996) 695–750. Zbl0867.60043MR1410112
  15. [15] H. Djellout, A. Guillin and L. Wu. Transportation cost-information inequalities and applications to random dynamical systems and diffusions. Ann. Probab. 32 (2004) 2702–2732. Zbl1061.60011MR2078555
  16. [16] B. S. Everitt and D. J. Hand. Finite Mixture Distributions. Chapman & Hall, London, 1981. Zbl0466.62018MR624267
  17. [17] V. P. Fonf, J. Lindenstrauss and R. R. Phelps. Infinite dimensional convexity. In Handbook of the Geometry of Banach Spaces 599–670. North-Holland, Amsterdam, 2001. Zbl1086.46004MR1863703
  18. [18] S. Frühwirth-Schnatter. Finite Mixture and Markov Switching Models. Springer, New York, 2006. Zbl1108.62002MR2265601
  19. [19] I. Gentil. Inégalités de Sobolev logarithmique et de Poincaré pour la loi uniforme. Unpublished 2004. Available on the author’s web page. 
  20. [20] I. Gentil and C. Roberto. Spectral gaps for spin systems: Some non-convex phase examples. J. Funct. Anal. 180 (2001) 66–84. Zbl0992.60091MR1814423
  21. [21] N. Gozlan. A characterization of dimension free concentration in terms of transportation inequalities. Preprint, 2008. Available at arXiv: 0804.3089 [math.PR]. Zbl1201.60016MR2573565
  22. [22] M. Gromov and V. D. Milman. A topological application of the isoperimetric inequality. Amer. J. Math. 105 (1983) 843–854. Zbl0522.53039MR708367
  23. [23] L. Gross. Logarithmic Sobolev inequalities. Amer. J. Math. 97 (1975) 1061–1083. Zbl0318.46049MR420249
  24. [24] L. Gross. Hypercontractivity, logarithmic Sobolev inequalities, and applications: A survey of surveys. In Diffusion, Quantum Theory, and Radically Elementary Mathematics 45–73. Math. Notes 47. Princeton Univ. Press, Princeton, NJ, 2006. MR2325763
  25. [25] B. Helffer. Semiclassical Analysis, Witten Laplacians, and Statistical Mechanics. Series in Partial Differential Equations and Applications 1. World Scientific Publishing, River Edge, NJ, 2002. Zbl1046.82001MR1936110
  26. [26] W. Hoeffding. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 (1963) 13–30. Zbl0127.10602MR144363
  27. [27] R. Holley and D. Stroock. Logarithmic Sobolev inequalities and stochastic Ising models. J. Statist. Phys. 46 (1987) 1159–1194. Zbl0682.60109MR893137
  28. [28] M. Jerrum, J.-B. Son, P. Tetali and E. Vigoda. Elementary bounds on Poincaré and log-Sobolev constants for decomposable Markov chains. Ann. Appl. Probab. 14 (2004) 1741–1765. Zbl1067.60065MR2099650
  29. [29] O. Johnson. Convergence of the Poincaré constant. Teor. Veroyatnost. i Primenen. 48 (2003) 615–620. Zbl1054.60026MR2141356
  30. [30] I. Kontoyiannis and M. Madiman. Measure concentration for compound Poisson distributions. Electron. Comm. Probab. 11 (2006) 45–57. Zbl1112.60008MR2219345
  31. [31] L. Kontoyiannis and M. Madiman. Entropy, compound Poisson approximation, log-Sobolev inequalities and measure concentration. In Information Theory Workshop, 24–29 Oct. 2004. IEEE 71–75, 2004. 
  32. [32] R. Latała. On some inequalities for Gaussian measures. In Proceedings of the International Congress of Mathematicians, Vol. II Beijing, 2002813–822. Higher Ed. Press, Beijing, 2002. Zbl1015.60011MR1957087
  33. [33] M. Ledoux. Concentration of measure and logarithmic Sobolev inequalities. In Séminaire de Probabilités, XXXIII 120–216. Springer, Berlin, 1999. Zbl0957.60016MR1767995
  34. [34] M. Ledoux. The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI, 2001. Zbl0995.60002MR1849347
  35. [35] N. Madras and D. Randall. Markov chain decomposition for convergence rate analysis. Ann. Appl. Probab. 12 (2002) 581–606. Zbl1017.60080MR1910641
  36. [36] K. Marton. A simple proof of the blowing-up lemma. IEEE Trans. Inform. Theory 32 (1986) 445–446. Zbl0594.94003MR838213
  37. [37] K. Marton. Bounding ̅d-distance by informational divergence: A method to prove measure concentration. Ann. Probab. 24 (1996) 857–866. Zbl0865.60017MR1404531
  38. [38] V. G. Maz’ja. Sobolev Spaces. Springer, Berlin, 1985. (Translated from the Russian by T. O. Shaposhnikova.) Zbl0692.46023MR817985
  39. [39] G. McLachlan and K. Basford. Mixture Models. Statistics: Textbooks and Monographs 84. Marcel Dekker Inc., New York, 1988. Zbl0697.62050MR926484
  40. [40] G. McLachlan and D. Peel. Finite Mixture Models. Wiley, New York, 2000. Zbl0963.62061MR1789474
  41. [41] L Miclo. Quand est-ce que des bornes de Hardy permettent de calculer une constante de Poincaré exacte sur la droite?. Preprint, 2005. Available at http://hal.archives-ouvertes.fr/hal-00017875/en/. MR2464097
  42. [42] F. Otto and M. G. Reznikoff. A new criterion for the logarithmic Sobolev inequality and two applications. J. Funct. Anal. 243 (2007) 121–157. Zbl1109.60013MR2291434
  43. [43] R. R. Phelps, Lectures on Choquet’s Theorem, 2nd edition. Lecture Notes in Math. 1757. Springer, Berlin, 2001. Zbl0997.46005MR1835574
  44. [44] S. T. Rachev. Probability Metrics and the Stability of Stochastic Models. Wiley, Chichester, 1991. Zbl0744.60004MR1105086
  45. [45] L. Saloff-Coste. Lectures on finite Markov chains. In Lectures on Probability Theory and Statistics (Saint-Flour, 1996) 301–413. Lecture Notes in Math. 1665. Springer, Berlin, 1997. Zbl0885.60061MR1490046
  46. [46] A. J. Stam. Some inequalities satisfied by the quantities of information of Fisher and Shannon. Information and Control 2 (1959) 101–112. Zbl0085.34701MR109101
  47. [47] V. N. Sudakov. Geometric problems in the theory of infinite-dimensional probability distributions. Proc. Steklov Inst. Math. (1979) 1–178. (Cover to cover translation of Trudy Mat. Inst. Steklov 141 (1976).) Zbl0409.60005MR530375
  48. [48] A. Takatsu. On Wasserstein geometry of the space of Gaussian measures, 2008. Available at arXiv:0801.2250 [math.DG]. 
  49. [49] M. Talagrand. Transportation cost for Gaussian and other product measures. Geom. Funct. Anal. 6 (1996) 587–600. Zbl0859.46030MR1392331
  50. [50] D. M. Titterington, A. F. M. Smith and U. E. Makov. Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester, 1985. Zbl0646.62013MR838090
  51. [51] C. Villani. Topics in Optimal Transportation. Graduate Studies in Mathematics 58. Amer. Math. Soc., Providence, RI, 2003. Zbl1106.90001MR1964483
  52. [52] C. Villani. Optimal Transport, Old and New. Springer, Berlin, 2009. Zbl1156.53003MR2459454

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.