The paper presents a new (to the best of the authors' knowledge) estimator of probability called the "Epₕ√2 completeness estimator" along with a theoretical derivation of its optimality. The estimator is especially suitable for a small number of sample items, which is the feature of many real problems characterized by data insufficiency. The control parameter of the estimator is not assumed in an a priori, subjective way, but was determined on the basis of an optimization criterion (the least absolute...

We establish the optimal quantization problem for probabilities under constrained Rényi-$\alpha$-entropy of the quantizers. We determine the optimal quantizers and the optimal quantization error of one-dimensional uniform distributions including the known special cases $\alpha =0$ (restricted codebook size) and $\alpha =1$ (restricted Shannon entropy).

In this paper we analyse a class of DASP (Digital Alias-free Signal Processing) methods for spectrum estimation of sampled signals. These methods consist in sampling the processed signals at randomly selected time instants. We construct estimators of Fourier transforms of the analysed signals. The estimators are unbiased inside arbitrarily wide frequency ranges, regardless of how sparsely the signal samples are collected. In order to facilitate quality assessment of the estimators, we calculate...

The information divergence of a probability measure $P$ from an exponential family $ℰ$ over a finite set is defined as infimum of the divergences of $P$ from $Q$ subject to $Q\in ℰ$. All directional derivatives of the divergence from $ℰ$ are explicitly found. To this end, behaviour of the conjugate of a log-Laplace transform on the boundary of its domain is analysed. The first order conditions for $P$ to be a maximizer of the divergence from $ℰ$ are presented, including new ones when $P$ is not projectable to $ℰ$.

We consider digit expansions ${\sum }_{j=0}^{\ell -1}{\Phi }^j\left(d_j\right)$ with an endomorphism $\Phi$ of an Abelian group. In such a numeral system, the $w$-NAF condition (each block of $w$ consecutive digits contains at most one nonzero) is shown to minimise the Hamming weight over all expansions with the same digit set if and only if it fulfills the subadditivity condition (the sum of every two expansions of weight $1$ admits an optimal $w$-NAF).This result is then applied to imaginary quadratic bases, which are used for scalar multiplication in elliptic...

This article studies exponential families $ℰ$ on finite sets such that the information divergence $D(P\parallel ℰ)$ of an arbitrary probability distribution from $ℰ$ is bounded by some constant $D>0$. A particular class of low-dimensional exponential families that have low values of $D$ can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. The case where $D=log\left(2\right)$ is studied in detail. This case is special, because if $D<log\left(2\right)$, then $ℰ$ contains all probability...

K. M. Wong and S. Chen [9] analyzed the Shannon entropy of a sequence of random variables under order restrictions. Using $(r,s)$-entropies, I. J. Taneja [8], these results are generalized. Upper and lower bounds to the entropy reduction when the sequence is ordered and conditions under which they are achieved are derived. Theorems are presented showing the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed...