Convexity inequalities for estimating generalized conditional entropies from below

Alexey E. Rastegin

Kybernetika (2012)

  • Volume: 48, Issue: 2, page 242-253
  • ISSN: 0023-5954

Abstract

top
Generalized entropic functionals are in an active area of research. Hence lower and upper bounds on these functionals are of interest. Lower bounds for estimating Rényi conditional α -entropy and two kinds of non-extensive conditional α -entropy are obtained. These bounds are expressed in terms of error probability of the standard decision and extend the inequalities known for the regular conditional entropy. The presented inequalities are mainly based on the convexity of some functions. In a certain sense, they are complementary to generalized inequalities of Fano type.

How to cite

top

Rastegin, Alexey E.. "Convexity inequalities for estimating generalized conditional entropies from below." Kybernetika 48.2 (2012): 242-253. <http://eudml.org/doc/246939>.

@article{Rastegin2012,
abstract = {Generalized entropic functionals are in an active area of research. Hence lower and upper bounds on these functionals are of interest. Lower bounds for estimating Rényi conditional $\alpha $-entropy and two kinds of non-extensive conditional $\alpha $-entropy are obtained. These bounds are expressed in terms of error probability of the standard decision and extend the inequalities known for the regular conditional entropy. The presented inequalities are mainly based on the convexity of some functions. In a certain sense, they are complementary to generalized inequalities of Fano type.},
author = {Rastegin, Alexey E.},
journal = {Kybernetika},
keywords = {Rènyi $\alpha $-entropy; non-extensive entropy of degree $\alpha $; error probability; Bayesian problems; functional convexity; Rényi -entropy; non-extensive entropy; error probability; Bayesian problems; functional convexity; generalized entropic functionals},
language = {eng},
number = {2},
pages = {242-253},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Convexity inequalities for estimating generalized conditional entropies from below},
url = {http://eudml.org/doc/246939},
volume = {48},
year = {2012},
}

TY - JOUR
AU - Rastegin, Alexey E.
TI - Convexity inequalities for estimating generalized conditional entropies from below
JO - Kybernetika
PY - 2012
PB - Institute of Information Theory and Automation AS CR
VL - 48
IS - 2
SP - 242
EP - 253
AB - Generalized entropic functionals are in an active area of research. Hence lower and upper bounds on these functionals are of interest. Lower bounds for estimating Rényi conditional $\alpha $-entropy and two kinds of non-extensive conditional $\alpha $-entropy are obtained. These bounds are expressed in terms of error probability of the standard decision and extend the inequalities known for the regular conditional entropy. The presented inequalities are mainly based on the convexity of some functions. In a certain sense, they are complementary to generalized inequalities of Fano type.
LA - eng
KW - Rènyi $\alpha $-entropy; non-extensive entropy of degree $\alpha $; error probability; Bayesian problems; functional convexity; Rényi -entropy; non-extensive entropy; error probability; Bayesian problems; functional convexity; generalized entropic functionals
UR - http://eudml.org/doc/246939
ER -

References

top
  1. L. Baladová, Minimum of average conditional entropy for given minimum probability of error., Kybernetika 2 (1966), 416-422. Zbl0199.21502MR0215641
  2. T. Cover, J. Thomas, Elements of Information Theory., John Wiley & Sons, New York 1991. Zbl1140.94001MR1122806
  3. I. Csiszár, 10.3390/e10030261, Entropy 10 (2008), 261-273. Zbl1179.94043DOI10.3390/e10030261
  4. Z. Daróczy, 10.1016/S0019-9958(70)80040-7, Inform. and Control 16 (1970), 36-51. Zbl0205.46901MR0272528DOI10.1016/S0019-9958(70)80040-7
  5. M. H. DeGroot, Optimal Statistical Decisions., McGraw-Hill, New York 1970. Zbl1136.62011MR0356303
  6. D. Erdogmus, J. C. Principe, 10.1023/B:VLSI.0000027493.48841.39, J. VLSI Signal Process. 37 (2004), 305-317. Zbl1073.94507DOI10.1023/B:VLSI.0000027493.48841.39
  7. R. M. Fano, Transmission of Information: A Statistical Theory of Communications., MIT Press and John Wiley & Sons, New York 1961. Zbl0151.24402MR0134389
  8. M. Feder, N. Merhav, 10.1109/18.272494, IEEE Trans. Inform. Theory 40 (1994), 259-266. Zbl0802.94004DOI10.1109/18.272494
  9. S. Furuichi, 10.1063/1.2165744, J. Math. Phys. 47 (2006), 023302. Zbl1111.94008MR2208160DOI10.1063/1.2165744
  10. M. Gell-Mann, C. Tsallis, eds., Nonextensive Entropy - Interdisciplinary Applications., Oxford University Press, Oxford 2004. Zbl1127.82004MR2073730
  11. G. H. Hardy, J. E. Littlewood, G. Polya, Inequalities., Cambridge University Press, London 1934. Zbl0634.26008
  12. J. Havrda, F. Charvát, Quantification methods of classification processes: concept of structural α -entropy., Kybernetika 3 (1967), 30-35. MR0209067
  13. P. Jizba, T. Arimitsu, 10.1016/j.aop.2004.01.002, Ann. Phys. 312 (2004), 17-59. Zbl1044.82001MR2067083DOI10.1016/j.aop.2004.01.002
  14. R. Kamimura, 10.1007/PL00013828, Algorithmica 22 (1998), 173-197. Zbl0910.68173MR1637503DOI10.1007/PL00013828
  15. A. Novikov, Optimal sequential procedures with Bayes decision rules., Kybernetika 46 (2010), 754-770. Zbl1201.62095MR2722099
  16. A. Perez, Information-theoretic risk estimates in statistical decision., Kybernetika 3 (1967), 1-21. Zbl0153.48403MR0208775
  17. A. E. Rastegin, 10.1088/1751-8113/43/15/155302, J. Phys. A: Math. Theor. 43 (2010), 155302. Zbl1189.81012MR2608279DOI10.1088/1751-8113/43/15/155302
  18. A. E. Rastegin, 10.1088/1751-8113/44/9/095303, J. Phys. A: Math. Theor. 44 (2011), 095303. Zbl1211.81021MR2771869DOI10.1088/1751-8113/44/9/095303
  19. A. E. Rastegin, Continuity estimates on the Tsallis relative entropy., E-print arXiv:1102.5154v2 [math-ph] (2011). MR2841748
  20. A. E. Rastegin, Fano type quantum inequalities in terms of q -entropies., Quantum Information Processing (2011), doi 10.1007/s11128-011-0347-6. 
  21. A. Rényi, On measures of entropy and information., In: Proc. 4th Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley - Los Angeles 1961, pp. 547-561. Zbl0106.33001MR0132570
  22. A. Rényi, On the amount of missing information in a random variable concerning an event., J. Math. Sci. 1 (1966), 30-33. MR0210263
  23. A. Rényi, Statistics and information theory., Stud. Sci. Math. Hung. 2 (1967), 249-256. Zbl0155.27602MR0212964
  24. A. Rényi, On some basic problems of statistics from the point of view of information theory., In: Proc. 5th Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley - Los Angeles 1967, pp. 531-543. Zbl0201.51905MR0212963
  25. B. Schumacher, 10.1103/PhysRevA.54.2614, Phys. Rev. A 54 (1996), 2614-2628. DOI10.1103/PhysRevA.54.2614
  26. C. Tsallis, 10.1007/BF01016429, J. Stat. Phys. 52 (1988), 479-487. Zbl1082.82501MR0968597DOI10.1007/BF01016429
  27. I. Vajda, On the statistical decision problem with discrete paprameter space., Kybernetika 3 (1967), 110-126. MR0215428
  28. I. Vajda, Bounds of the minimal error probability on checking a finite or countable number of hypotheses., Problemy Peredachii Informacii 4 (1968), 9-19 (in Russian); translated as Problems of Information Transmission 4 (1968), 6-14. MR0267685
  29. K. Życzkowski, Rényi extrapolation of Shannon entropy., Open Sys. Inform. Dyn. 10 (2003), 297-310; corrigendum in the e-print version arXiv:quant-ph/0305062v2. Zbl1030.94022MR1998623

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.