Belis and Guiasu studied a generalization of Shannon entropy as weighted or useful entropy. In this paper, the weighted entropy of type $(\alpha ,\beta )$ is defined and characterized and some if its properties are studied. Further generalizations involving more parameters of weighted entropy are also specified.

We establish the optimal quantization problem for probabilities under constrained Rényi-$\alpha$-entropy of the quantizers. We determine the optimal quantizers and the optimal quantization error of one-dimensional uniform distributions including the known special cases $\alpha =0$ (restricted codebook size) and $\alpha =1$ (restricted Shannon entropy).

The information divergence of a probability measure $P$ from an exponential family $ℰ$ over a finite set is defined as infimum of the divergences of $P$ from $Q$ subject to $Q\in ℰ$. All directional derivatives of the divergence from $ℰ$ are explicitly found. To this end, behaviour of the conjugate of a log-Laplace transform on the boundary of its domain is analysed. The first order conditions for $P$ to be a maximizer of the divergence from $ℰ$ are presented, including new ones when $P$ is not projectable to $ℰ$.

This article studies exponential families $ℰ$ on finite sets such that the information divergence $D(P\parallel ℰ)$ of an arbitrary probability distribution from $ℰ$ is bounded by some constant $D>0$. A particular class of low-dimensional exponential families that have low values of $D$ can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. The case where $D=log\left(2\right)$ is studied in detail. This case is special, because if $D<log\left(2\right)$, then $ℰ$ contains all probability...