Bounds for f -divergences under likelihood ratio constraints

Sever Silvestru Dragomir

Applications of Mathematics (2003)

  • Volume: 48, Issue: 3, page 205-223
  • ISSN: 0862-7940

Abstract

top
In this paper we establish an upper and a lower bound for the f -divergence of two discrete random variables under likelihood ratio constraints in terms of the Kullback-Leibler distance. Some particular cases for Hellinger and triangular discimination, χ 2 -distance and Rényi’s divergences, etc. are also considered.

How to cite

top

Dragomir, Sever Silvestru. "Bounds for $f$-divergences under likelihood ratio constraints." Applications of Mathematics 48.3 (2003): 205-223. <http://eudml.org/doc/33145>.

@article{Dragomir2003,
abstract = {In this paper we establish an upper and a lower bound for the $f$-divergence of two discrete random variables under likelihood ratio constraints in terms of the Kullback-Leibler distance. Some particular cases for Hellinger and triangular discimination, $\chi ^2$-distance and Rényi’s divergences, etc. are also considered.},
author = {Dragomir, Sever Silvestru},
journal = {Applications of Mathematics},
keywords = {$f$-divergence; divergence measures in information theory; Jensen’s inequality; Hellinger and triangular discrimination; -divergence; divergence measures in information theory; Jensen's inequality; Hellinger and triangular discrimination},
language = {eng},
number = {3},
pages = {205-223},
publisher = {Institute of Mathematics, Academy of Sciences of the Czech Republic},
title = {Bounds for $f$-divergences under likelihood ratio constraints},
url = {http://eudml.org/doc/33145},
volume = {48},
year = {2003},
}

TY - JOUR
AU - Dragomir, Sever Silvestru
TI - Bounds for $f$-divergences under likelihood ratio constraints
JO - Applications of Mathematics
PY - 2003
PB - Institute of Mathematics, Academy of Sciences of the Czech Republic
VL - 48
IS - 3
SP - 205
EP - 223
AB - In this paper we establish an upper and a lower bound for the $f$-divergence of two discrete random variables under likelihood ratio constraints in terms of the Kullback-Leibler distance. Some particular cases for Hellinger and triangular discimination, $\chi ^2$-distance and Rényi’s divergences, etc. are also considered.
LA - eng
KW - $f$-divergence; divergence measures in information theory; Jensen’s inequality; Hellinger and triangular discrimination; -divergence; divergence measures in information theory; Jensen's inequality; Hellinger and triangular discrimination
UR - http://eudml.org/doc/33145
ER -

References

top
  1. Information measures: A critical survey, Trans. 7th Prague Conf. on Info. Th., Statist. Decis. Funct., Random Processes and 8th European Meeting of Statist., Volume  B, Academia Prague, 1978, pp. 73–86. (1978) MR0519465
  2. Information-type measures of difference of probability functions and indirect observations, Studia Sci. Math. Hungar. 2 (1967), 299–318. (1967) MR0219345
  3. Information Theory: Coding Theorem for Discrete Memory-less Systems, Academic Press, New York, 1981. (1981) MR0666545
  4. Maximum Entropy and Bayesian Methods in Applied Statistics, J. H.  Justice (ed.), Cambridge University Press, Cambridge, 1986. (1986) Zbl0597.00025MR0892126
  5. On the roles of maximum entropy and minimum discrimination information principles in Statistics, Technical Address of the 38th Annual Conference of the Indian Society of Agricultural Statistics, 1984, pp. 1–44. (1984) MR0786009
  6. 10.1109/TIT.1982.1056497, IEEE Transactions on Information Theory 28 (1982), 489–495. (1982) MR0672884DOI10.1109/TIT.1982.1056497
  7. Information Theory and Reliable Communications, J.  Wiley, New York, 1968. (1968) 
  8. A mathematical theory of communication, Bull. Sept. Tech. J. 27 (1948), 370–423 and 623–656. (1948) Zbl1154.94303MR0026286
  9. Image enhancement and restoration, Picture Processing and Digital Filtering, T. S.  Huang (ed.), Springer-Verlag, Berlin, 1975. (1975) 
  10. 10.1109/TASSP.1986.1165001, IEEE Trans. on Acoustics, Speech and Signal Processing 34 (1986), 1629–1642. (1986) DOI10.1109/TASSP.1986.1165001
  11. Information Theory and Statistics, J.  Wiley, New York, 1959. (1959) Zbl0088.10406MR0103557
  12. 10.1214/aoms/1177729694, Ann. Math. Statistics 22 (1951), 79–86. (1951) MR0039968DOI10.1214/aoms/1177729694
  13. 10.1214/aos/1176343842, Ann. Statist. 5 (1977), 445–463. (1977) Zbl0381.62028MR0448700DOI10.1214/aos/1176343842
  14. On measures of entropy and information, Proc. Fourth Berkeley Symp. Math. Statist. Prob., Vol. 1, University of California Press, Berkeley, 1961. (1961) Zbl0106.33001MR0132570
  15. Some converse of Jensen’s inequality and applications, Anal. Numer. Theor. Approx. 23 (1994), 71–78. (1994) MR1325895
  16. 10.1016/0895-7177(96)00085-4, Math. Comput. Modelling 24 (1996), 1–11. (1996) MR1403525DOI10.1016/0895-7177(96)00085-4
  17. Some counterpart inequalities in for a functional associated with Jensen’s inequality, J. Inequal. Appl. 1 (1997), 311–325. (1997) MR1732628
  18. 10.1016/S0893-9659(97)00028-1, Appl. Math. Lett. 10 (1997), 23–28. (1997) MR1457634DOI10.1016/S0893-9659(97)00028-1
  19. A counterpart of Jensen’s continuous inequality and applications in information theory, RGMIA Preprint.http://matilda.vu.edu.au/~rgmia/InfTheory/Continuse.dvi. MR1977385
  20. Jensen’s inequality and applications in information theory, Ph.D. Thesis, Univ. of Zagreb, 1999. (Croatian) (1999) 
  21. 10.1016/S0898-1221(00)00089-4, Comput. Math. Appl. 39 (2000), 257–266. (2000) MR1753564DOI10.1016/S0898-1221(00)00089-4
  22. A comparative assessment of various measures of directed divergence, Advances Manag. Stud. 3 (1984), 1–16. (1984) 
  23. Classical and New Inequalities in Analysis, Kluwer Academic Publishers, , 1993. (1993) MR1220224
  24. Some inequalities for information divergence and related measures of discrimination, Res. Rep. Coll. RGMIA 2 (1999), 85–98. (1999) MR1768575
  25. Vitesse de convergence pour certains problemes statistiques, In: Ecole d’été de Probabilités de Saint-Flour, VII (Saint-Flour, 1977). Lecture Notes in Math. Vol. 678, Springer, Berlin, 1978, pp. 1–172. (1978) Zbl0387.62015MR0518733
  26. 10.1098/rspa.1946.0056, Proc. Roy. Soc. London Ser.  A 186 (1946), 453–461. (1946) Zbl0063.03050MR0017504DOI10.1098/rspa.1946.0056
  27. On a measure of divergence between two statistical populations defined by their probability distributions, Bull. Calcutta Math. Soc. 35 (1943), 99–109. (1943) Zbl0063.00364MR0010358
  28. Some inequalities for the Csiszár Φ -divergence, Submitted. 
  29. A converse inequality for the Csiszár Φ -divergence, Submitted. Zbl1066.94007
  30. Some inequalities for ( m , M ) -convex mappings and applications for the Csiszár Φ -divergence in information theory, Math. J. Ibaraki Univ. (Japan) 33 (2001), 35–50. (2001) Zbl0996.26019MR1883258
  31. Convex Statistical Distances, Teubner Verlag, Leipzig, 1987. (1987) MR0926905
  32. Theory of Statistical Inference and Information, Kluwer, Boston, 1989. (1989) Zbl0711.62002

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.