φ PHI-divergences, sufficiency, Bayes sufficiency, and deficiency

Friedrich Liese

Kybernetika (2012)

  • Volume: 48, Issue: 4, page 690-713
  • ISSN: 0023-5954

Abstract

top
The paper studies the relations between φ -divergences and fundamental concepts of decision theory such as sufficiency, Bayes sufficiency, and LeCam’s deficiency. A new and considerably simplified approach is given to the spectral representation of φ -divergences already established in Österreicher and Feldman [28] under restrictive conditions and in Liese and Vajda [22], [23] in the general form. The simplification is achieved by a new integral representation of convex functions in terms of elementary convex functions which are strictly convex at one point only. Bayes sufficiency is characterized with the help of a binary model that consists of the joint distribution and the product of the marginal distributions of the observation and the parameter, respectively. LeCam’s deficiency is expressed in terms of φ -divergences where φ belongs to a class of convex functions whose curvature measures are finite and satisfy a normalization condition.

How to cite

top

Liese, Friedrich. "$\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency." Kybernetika 48.4 (2012): 690-713. <http://eudml.org/doc/246799>.

@article{Liese2012,
abstract = {The paper studies the relations between $\phi $-divergences and fundamental concepts of decision theory such as sufficiency, Bayes sufficiency, and LeCam’s deficiency. A new and considerably simplified approach is given to the spectral representation of $\phi $-divergences already established in Österreicher and Feldman [28] under restrictive conditions and in Liese and Vajda [22], [23] in the general form. The simplification is achieved by a new integral representation of convex functions in terms of elementary convex functions which are strictly convex at one point only. Bayes sufficiency is characterized with the help of a binary model that consists of the joint distribution and the product of the marginal distributions of the observation and the parameter, respectively. LeCam’s deficiency is expressed in terms of $\phi $-divergences where $\phi $ belongs to a class of convex functions whose curvature measures are finite and satisfy a normalization condition.},
author = {Liese, Friedrich},
journal = {Kybernetika},
keywords = {divergences; sufficiency; Bayes sufficiency; deficiency; divergences; sufficiency; Bayes sufficiency; deficiency},
language = {eng},
number = {4},
pages = {690-713},
publisher = {Institute of Information Theory and Automation AS CR},
title = {$\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency},
url = {http://eudml.org/doc/246799},
volume = {48},
year = {2012},
}

TY - JOUR
AU - Liese, Friedrich
TI - $\phi $PHI-divergences, sufficiency, Bayes sufficiency, and deficiency
JO - Kybernetika
PY - 2012
PB - Institute of Information Theory and Automation AS CR
VL - 48
IS - 4
SP - 690
EP - 713
AB - The paper studies the relations between $\phi $-divergences and fundamental concepts of decision theory such as sufficiency, Bayes sufficiency, and LeCam’s deficiency. A new and considerably simplified approach is given to the spectral representation of $\phi $-divergences already established in Österreicher and Feldman [28] under restrictive conditions and in Liese and Vajda [22], [23] in the general form. The simplification is achieved by a new integral representation of convex functions in terms of elementary convex functions which are strictly convex at one point only. Bayes sufficiency is characterized with the help of a binary model that consists of the joint distribution and the product of the marginal distributions of the observation and the parameter, respectively. LeCam’s deficiency is expressed in terms of $\phi $-divergences where $\phi $ belongs to a class of convex functions whose curvature measures are finite and satisfy a normalization condition.
LA - eng
KW - divergences; sufficiency; Bayes sufficiency; deficiency; divergences; sufficiency; Bayes sufficiency; deficiency
UR - http://eudml.org/doc/246799
ER -

References

top
  1. M. S. Ali, D. Silvey, A general class of coefficients of divergence of one distribution from another., J. Roy. Statist. Soc. Ser. B 28 (1966), 131-140. Zbl0203.19902MR0196777
  2. S. Arimoto, 10.1016/S0019-9958(71)90065-9, Inform. Control. 19 (1971), 181-194. Zbl0222.94022MR0309224DOI10.1016/S0019-9958(71)90065-9
  3. A. R. Barron, L. Györfi, E. C. van der Meulen, 10.1109/18.149496, IEEE Trans. Inform. Theory 38 (1990), 1437-1454. DOI10.1109/18.149496
  4. A. Berlinet, I. Vajda, E. C. van der Meulen, 10.1109/18.669143, IEEE Trans. Inform. Theory 44 (1990), 999-1009. Zbl0952.62029MR1616679DOI10.1109/18.669143
  5. A. Bhattacharyya, On some analogues to the amount of information and their uses in statistical estimation., Sankhya 8 (1946), 1-14. MR0020242
  6. H. Chernoff, 10.1214/aoms/1177729330, Ann. Math. Statist. 23 (1952), 493-507. MR0057518DOI10.1214/aoms/1177729330
  7. B. S. Clarke, A. R. Barron, 10.1109/18.54897, IEEE Trans. Inform. Theory 36 (1990), 453-471. Zbl0709.62008MR1053841DOI10.1109/18.54897
  8. I. Csiszár, Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffscher Ketten., Publ. Math. Inst. Hungar. Acad. Sci.8 (1963), 84-108. MR0164374
  9. I. Csiszár, Information-type measures of difference of probability distributions and indirect observations., Studia Sci. Math. Hungar. 2, (1967), 299-318. Zbl0157.25802MR0219345
  10. T. Cover, J. Thomas, Elements of Information Theory., Wiley, New York 1991. Zbl1140.94001MR1122806
  11. M. H. De Groot, Optimal Statistical Decisions., McGraw Hill, New York 1970. MR0356303
  12. D. Feldman, F. Österreicher, A note on f -divergences., Studia Sci. Math. Hungar. 24 (1989), 191-200. Zbl0725.62005MR1051149
  13. A. Guntuboyina, 10.1109/TIT.2011.2110791, IEEE Trans. Inform. Theory 57 (2011), 2386-2399. MR2809097DOI10.1109/TIT.2011.2110791
  14. C. Guttenbrunner, On applications of the representation of f -divergences as averaged minimal Bayesian risk., In: Trans. 11th Prague Conf. Inform. Theory, Statist. Dec. Funct., Random Processes A, 1992, pp. 449-456. 
  15. L. Jager, J. A. Wellner, 10.1214/0009053607000000244, Ann. Statist. 35 (2007), 2018-2053. Zbl1126.62030MR2363962DOI10.1214/0009053607000000244
  16. T. Kailath, 10.1109/TCOM.1967.1089532, IEEE Trans. Commun. Technol. 15 (1990), 52-60. DOI10.1109/TCOM.1967.1089532
  17. S. Kakutani, 10.2307/1969123, Ann. Math. 49 (1948), 214-224. Zbl0030.02303MR0023331DOI10.2307/1969123
  18. S. Kullback, R. Leibler, 10.1214/aoms/1177729694, Ann. Math. Statist. 22 (1951), 79-86. Zbl0042.38403MR0039968DOI10.1214/aoms/1177729694
  19. L. LeCam, Locally asymptotically normal families of distributions., Univ. Calif. Publ. 3, (1960), 37-98. MR0126903
  20. L. LeCam, Asymptotic Methods in Statistical Decision Theory., Springer, Berlin 1986. 
  21. F. Liese, I. Vajda, Convex Statistical Distances., Teubner, Leipzig 1987. Zbl0656.62004MR0926905
  22. F. Liese, I. Vajda, 10.1109/TIT.2006.881731, IEEE Trans. Inform. Theory 52 (2006), 4394-4412. MR2300826DOI10.1109/TIT.2006.881731
  23. F. Liese, I. Vajda, f -divergences: Sufficiency, deficiency and testing of gypotheses., In: Advances in Inequalities from Probability Theory and Statistics. (N. S. Barnett and S. S. Dragomir, eds.), Nova Science Publisher, Inc., New York 2008, pp. 113-149. MR2459971
  24. F. Liese, K. J. Miescke, Statistical Decision Theory, Estimation, Testing and Selection., Springer, New York 2008. Zbl1154.62008MR2421720
  25. K. Matusita, 10.1214/aoms/1177728422, Ann. Math. Statist. 26 (1955), 613-640. Zbl0065.12101MR0073899DOI10.1214/aoms/1177728422
  26. D. Mussmann, Decision rules based on the distance, for problems of fit, two samples and estimation., Studia Sci. Math. Hungar. 14 (1979), 37-41. 
  27. X. Nguyen, M. J. Wainwright, M. I. Jordan, 10.1214/08-AOS595, Ann. Statist. 37 (2009), 2018-2053. Zbl1162.62060MR2502654DOI10.1214/08-AOS595
  28. F. Österreicher, D. Feldman, 10.1007/BF01895132, Acta Math. Sci. Hungar. 37 (1981), 329-337. Zbl0477.60013MR0619882DOI10.1007/BF01895132
  29. F. Österreicher, I. Vajda, 10.1109/18.256536, IEEE Trans. Inform. Theory 39 (1993), 1036-1039. Zbl0792.62005MR1237725DOI10.1109/18.256536
  30. J. Pfanzagl, 10.1007/BF01893900, Metrika 21 (1974), 197-199. Zbl0289.62009MR0365797DOI10.1007/BF01893900
  31. H. V. Poor, Robust decision design using a distance criterion., IEEE Trans. Inform. Theory 26 (1980), 578-587. Zbl0445.62017MR0583942
  32. M. R. C. Read, N. A. C. Cressie, Goodness-of-Fit Statistics for Discrete Multivariate Data., Springer, Berlin 1988. Zbl0663.62065MR0955054
  33. A. Rényi, On measures of entropy and information., In: Proc. 4th Berkeley Symp. on Probab. Theory and Math. Statist. Berkeley Univ. Press, Berkeley 1961, pp. 547-561. Zbl0106.33001MR0132570
  34. A. W. Roberts, D. E. Varberg, Convex Functions., Academic Press, New York 1973. Zbl0289.26012MR0442824
  35. M. J. Schervish, Theory of Statistics., Springer, New York 1995. Zbl0834.62002MR1354146
  36. C. E. Shannon, 10.1002/j.1538-7305.1948.tb01338.x, Bell. Syst. Tech. J. 27 (1948), 379-423, 623-656. Zbl1154.94303MR0026286DOI10.1002/j.1538-7305.1948.tb01338.x
  37. H. Strasser, Mathematical Theory of Statistics., De Gruyter, Berlin 1985. Zbl0594.62017MR0812467
  38. F. Topsøe, Information-theoretical optimization techniques., Kybernetika 15 (1979), 7-17. MR0529888
  39. F. Topsøe, 10.1109/18.850703, IEEE Trans. Inform. Theory 46 (2000), 1602-1609. MR1768575DOI10.1109/18.850703
  40. E. Torgersen, Comparison of Statistical Experiments., Cambridge Univ. Press, Cambridge 1991. Zbl1158.62006MR1104437
  41. I. Vajda, 10.1007/BF02018663, Periodica Math. Hungar. 2 (1972), 223-234. Zbl0248.62001MR0335163DOI10.1007/BF02018663
  42. I. Vajda, Theory of Statistical Inference and Information., Kluwer Academic Publishers, Dordrecht - Boston - London 1989. Zbl0711.62002
  43. I. Vajda, On metric divergences of probability measures., Kybernetika 45 (2009), 885-900. MR2650071
  44. I. Vajda, 10.1109/TIT.2002.800497, IEEE Trans. Inform. Theory. 48 (1980) 2163-2172. Zbl1062.94533MR1930280DOI10.1109/TIT.2002.800497
  45. I. Vincze, On the concept and measure of information contained in an observation., In: Contribution to Probability. (J. Gani and V. F. Rohatgi, eds.) Academic Press, New York 1981, pp. 207-214. Zbl0531.62002MR0618690

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.