We investigate solution sets of a special kind of linear inequality systems. In particular, we derive characterizations of these sets in terms of minimal solution sets. The studied inequalities emerge as information inequalities in the context of Bayesian networks. This allows to deduce structural properties of Bayesian networks, which is important within causal inference.
In this series, this paper is devoted to the study of two related functional equations primarily connected with weighted entropy and weighted entropy of degree beta (which are weighted additive and weighted beta additive respectively) which include as special cases Shannon's entropy, inaccuracy (additive measures) and the entropy of degree beta (nonadditive) respectively. These functional equations which arise mainly from the representation and these 'additive' properties are solved for fixed m...
We present a mathematical model allowing formally define the concepts of empirical and theoretical knowledge. The model consists of a finite set P of predicates and a probability space (Ω, S, P) over a finite set Ω called ontology which consists of objects ω for which the predicates π ∈ P are either valid (π(ω) = 1) or not valid (π(ω) = 0). Since this is a first step in this area, our approach is as simple as possible, but still nontrivial, as it is demonstrated by examples. More realistic approach...
We address the problem of computing the capacity of a covert channel, modeled as a nondeterministic transducer. We give three possible statements of the notion of “covert channel capacity” and relate the different definitions. We then provide several methods allowing the computation of lower and upper bounds for the capacity of a channel. We show that, in some cases, including the case of input-deterministic channels, the capacity of the channel can be computed exactly (e.g. in the form...
The Hudetz correction of the fuzzy entropy is applied to the -entropy. The new invariant is expressed by the Hudetz correction of fuzzy entropy.
We establish a decomposition of the Jensen-Shannon divergence into a linear combination of a scaled Jeffreys' divergence and a reversed Jensen-Shannon divergence. Upper and lower bounds for the Jensen-Shannon divergence are then found in terms of the squared (total) variation distance. The derivations rely upon the Pinsker inequality and the reverse Pinsker inequality. We use these bounds to prove the asymptotic equivalence of the maximum likelihood estimate and minimum Jensen-Shannon divergence...
In this paper the mean and the variance of the Maximum Likelihood Estimator (MLE) of Kullback information measure and measure of relative "useful" information are obtained.