Definition and some Properties of Information Entropy

Bo Zhang; Yatsuka Nakamura

Formalized Mathematics (2007)

  • Volume: 15, Issue: 3, page 111-119
  • ISSN: 1426-2630

Abstract

top
In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon's lemma, maximum property, additivity and super-additivity properties.

How to cite

top

Bo Zhang, and Yatsuka Nakamura. "Definition and some Properties of Information Entropy." Formalized Mathematics 15.3 (2007): 111-119. <http://eudml.org/doc/267443>.

@article{BoZhang2007,
abstract = {In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon's lemma, maximum property, additivity and super-additivity properties.},
author = {Bo Zhang, Yatsuka Nakamura},
journal = {Formalized Mathematics},
language = {eng},
number = {3},
pages = {111-119},
title = {Definition and some Properties of Information Entropy},
url = {http://eudml.org/doc/267443},
volume = {15},
year = {2007},
}

TY - JOUR
AU - Bo Zhang
AU - Yatsuka Nakamura
TI - Definition and some Properties of Information Entropy
JO - Formalized Mathematics
PY - 2007
VL - 15
IS - 3
SP - 111
EP - 119
AB - In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon's lemma, maximum property, additivity and super-additivity properties.
LA - eng
UR - http://eudml.org/doc/267443
ER -

References

top
  1. [1] Grzegorz Bancerek. The ordinal numbers. Formalized Mathematics, 1(1):91-96, 1990. 
  2. [2] Grzegorz Bancerek and Krzysztof Hryniewiecki. Segments of natural numbers and finite sequences. Formalized Mathematics, 1(1):107-114, 1990. 
  3. [3] P. Billingsley. Ergodic Theory and Information. John Wiley & Sons, 1964. Zbl0184.43301
  4. [4] Czesław Byliński. Binary operations. Formalized Mathematics, 1(1):175-180, 1990. 
  5. [5] Czesław Byliński. Finite sequences and tuples of elements of a non-empty sets. Formalized Mathematics, 1(3):529-536, 1990. 
  6. [6] Czesław Byliński. Functions and their basic properties. Formalized Mathematics, 1(1):55-65, 1990. 
  7. [7] Czesław Byliński. Functions from a set to a set. Formalized Mathematics, 1(1):153-164, 1990. 
  8. [8] Czesław Byliński. Some basic properties of sets. Formalized Mathematics, 1(1):47-53, 1990. 
  9. [9] Czesław Byliński. The sum and product of finite sequences of real numbers. Formalized Mathematics, 1(4):661-668, 1990. 
  10. [10] Agata Darmochwał. The Euclidean space. Formalized Mathematics, 2(4):599-603, 1991. 
  11. [11] Shigeichi Hirasawa. Information Theory. Baifukan CO., 1996. 
  12. [12] Katarzyna Jankowska. Matrices. Abelian group of matrices. Formalized Mathematics, 2(4):475-480, 1991. 
  13. [13] Artur Korniłowicz. On the real valued functions. Formalized Mathematics, 13(1):181-187, 2005. 
  14. [14] Jarosław Kotowicz. Real sequences and basic operations on them. Formalized Mathematics, 1(2):269-272, 1990. 
  15. [15] Rafał Kwiatek. Factorial and Newton coefficients. Formalized Mathematics, 1(5):887-890, 1990. 
  16. [16] Yatsuka Nakamura, Nobuyuki Tamaura, and Wenpai Chang. A theory of matrices of real elements. Formalized Mathematics, 14(1):21-28, 2006. 
  17. [17] Library Committee of the Association of Mizar Users. Binary operations on numbers. To appear in Formalized Mathematics. 
  18. [18] Konrad Raczkowski and Andrzej Nedzusiak. Real exponents and logarithms. Formalized Mathematics, 2(2):213-216, 1991. 
  19. [19] Yasunari Shidama. The Taylor expansions. Formalized Mathematics, 12(2):195-200, 2004. 
  20. [20] Andrzej Trybulec. Subsets of complex numbers. To appear in Formalized Mathematics. 
  21. [21] Andrzej Trybulec. Tarski Grothendieck set theory. Formalized Mathematics, 1(1):9-11, 1990. 
  22. [22] Wojciech A. Trybulec. Vectors in real linear space. Formalized Mathematics, 1(2):291-296, 1990. 
  23. [23] Zinaida Trybulec. Properties of subsets. Formalized Mathematics, 1(1):67-71, 1990. 
  24. [24] Edmund Woronowicz. Relations and their basic properties. Formalized Mathematics, 1(1):73-83, 1990. 
  25. [25] Bo Zhang and Yatsuka Nakamura. The definition of finite sequences and matrices of probability, and addition of matrices of real elements. Formalized Mathematics, 14(3):101-108, 2006. 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.