Advanced Search

Stochastic interdependence of a probability distribution on a product space is measured by its Kullback–Leibler distance from the exponential family of product distributions (called multi-information). Here we investigate low-dimensional exponential families that contain the maximizers of stochastic interdependence in their closure. Based on a detailed description of the structure of probability distributions with globally maximal multi-information we obtain our main result: The exponential family...

We investigate solution sets of a special kind of linear inequality systems. In particular, we derive characterizations of these sets in terms of minimal solution sets. The studied inequalities emerge as information inequalities in the context of Bayesian networks. This allows to deduce structural properties of Bayesian networks, which is important within causal inference.

Given a fixed dependency graph $G$ that describes a Bayesian network of binary variables $X_1,\cdots ,X_n$, our main result is a tight bound on the mutual information $I_c(Y_1,\cdots ,Y_k)={\sum }_{j=1}^{k}H\left(Y_j\right)/c-H(Y_1,\cdots ,Y_k)$ of an observed subset $Y_1,\cdots ,Y_k$ of the variables $X_1,\cdots ,X_n$. Our bound depends on certain quantities that can be computed from the connective structure of the nodes in $G$. Thus it allows to discriminate between different dependency graphs for a probability distribution, as we show from numerical experiments.

In this paper, we explore a connection between binary hierarchical models, their marginal polytopes, and codeword polytopes, the convex hulls of linear codes. The class of linear codes that are realizable by hierarchical models is determined. We classify all full dimensional polytopes with the property that their vertices form a linear code and give an algorithm that determines them.