This article studies exponential families $ℰ$ on finite sets such that the information divergence $D(P\parallel ℰ)$ of an arbitrary probability distribution from $ℰ$ is bounded by some constant $D>0$. A particular class of low-dimensional exponential families that have low values of $D$ can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. The case where $D=log\left(2\right)$ is studied in detail. This case is special, because if $D<log\left(2\right)$, then $ℰ$ contains all probability...

K. M. Wong and S. Chen [9] analyzed the Shannon entropy of a sequence of random variables under order restrictions. Using $(r,s)$-entropies, I. J. Taneja [8], these results are generalized. Upper and lower bounds to the entropy reduction when the sequence is ordered and conditions under which they are achieved are derived. Theorems are presented showing the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed...