Properties of unique information

Johannes Rauh; Maik Schünemann; Jürgen Jost

Kybernetika (2021)

  • Volume: 57, Issue: 3, page 383-403
  • ISSN: 0023-5954

Abstract

top
We study the unique information function U I ( T : X ∖ Y ) defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of U I . We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of T , X and Y . Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of U I ( T : X ∖ Y ) , most notably when T is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if T is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie.

How to cite

top

Rauh, Johannes, Schünemann, Maik, and Jost, Jürgen. "Properties of unique information." Kybernetika 57.3 (2021): 383-403. <http://eudml.org/doc/297757>.

@article{Rauh2021,
abstract = {We study the unique information function $UI(T:X\setminus Y)$ defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if $T$ is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie.},
author = {Rauh, Johannes, Schünemann, Maik, Jost, Jürgen},
journal = {Kybernetika},
keywords = {information decomposition; unique information},
language = {eng},
number = {3},
pages = {383-403},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Properties of unique information},
url = {http://eudml.org/doc/297757},
volume = {57},
year = {2021},
}

TY - JOUR
AU - Rauh, Johannes
AU - Schünemann, Maik
AU - Jost, Jürgen
TI - Properties of unique information
JO - Kybernetika
PY - 2021
PB - Institute of Information Theory and Automation AS CR
VL - 57
IS - 3
SP - 383
EP - 403
AB - We study the unique information function $UI(T:X\setminus Y)$ defined by Bertschinger et al. within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if $T$ is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie.
LA - eng
KW - information decomposition; unique information
UR - http://eudml.org/doc/297757
ER -

References

top
  1. Amari, Shun-ichi, Nagaoka, Hiroshi, , American Mathematical Society 2000. DOI
  2. Banerjee, P. Kr., Olbrich, E., Jost, J., Rauh, J., , In: Proc. Allerton, 2018. DOI
  3. Banerjee, P. Kr., Rauh, J., Montúfar, G., , In: Proc. IEEE ISIT 2018, pp. 141-145 DOI
  4. Bertschinger, N., Rauh, J., Olbrich, E., Jost, J., Ay, N., , Entropy 16 (2014), 4, 2161-2183. DOI
  5. Diaconis, Persi, Sturmfels, Bernd, Algebraic algorithms for sampling from conditional distributions., Ann. Statist. 26 (1998), 363-397. 
  6. Fink, A., , J. Algebr. Combin. 33 (2011), 3, 455-463. DOI
  7. Finn, C., Lizier, J. T., , Entropy 20 (2018), 4, 297. DOI
  8. Harder, M., Salge, Ch., Polani, D., , Phys. Rev. E 87 (2013), 012130. DOI
  9. Ince, R., , Entropy 19 (2017), 7, 318. DOI
  10. James, R., Emenheiser, J., Crutchfield, J., , J. Physics A 52 (2018), 1, 014002. DOI
  11. Makkeh, A., Theis, D. O., Vicente, R., , Entropy 19 (2017), 10, 530. DOI
  12. Niu, X., Quinn, Ch., , In: Proc. IEEE ISIT 2019. DOI
  13. Rauh, J., Banerjee, P. Kr., Olbrich, E., Jost, J., , In: 2019 IEEE International Symposium on Information Theory (ISIT), pp. 3042-3046. DOI
  14. Rauh, J., Ay, N., , Theory Biosciences 133 (2014), 2, 63-78. DOI
  15. Rauh, J., Bertschinger, N., Olbrich, E., Jost, J., , In: Proc. IEEE ISIT 2014, pp. 2232-2236. DOI
  16. Smith, N. A., R, Tromble, W., Sampling Uniformly from the Unit Simplex., Technical Report 29, Johns Hopkins University 29, 2004. 
  17. Williams, P., Beer, R., Nonnegative decomposition of multivariate information., arXiv:1004.2515v1. 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.