Scaling of model approximation errors and expected entropy distances

Montúfar, Guido F., and Rauh, Johannes. "Scaling of model approximation errors and expected entropy distances." Kybernetika 50.2 (2014): 234-245. <http://eudml.org/doc/261861>.

@article{Montúfar2014,
abstract = {We compute the expected value of the Kullback-Leibler divergence of various fundamental statistical models with respect to Dirichlet priors. For the uniform prior, the expected divergence of any model containing the uniform distribution is bounded by a constant $1-\gamma $. For the models that we consider this bound is approached as the cardinality of the sample space tends to infinity, if the model dimension remains relatively small. For Dirichlet priors with reasonable concentration parameters the expected values of the divergence behave in a similar way. These results serve as a reference to rank the approximation capabilities of other statistical models.},
author = {Montúfar, Guido F., Rauh, Johannes},
journal = {Kybernetika},
keywords = {exponential families; KL divergence; MLE; Dirichlet prior; exponential families; KL divergence; MLE; Dirichlet prior},
language = {eng},
number = {2},
pages = {234-245},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Scaling of model approximation errors and expected entropy distances},
url = {http://eudml.org/doc/261861},
volume = {50},
year = {2014},
}

TY - JOUR
AU - Montúfar, Guido F.
AU - Rauh, Johannes
TI - Scaling of model approximation errors and expected entropy distances
JO - Kybernetika
PY - 2014
PB - Institute of Information Theory and Automation AS CR
VL - 50
IS - 2
SP - 234
EP - 245
AB - We compute the expected value of the Kullback-Leibler divergence of various fundamental statistical models with respect to Dirichlet priors. For the uniform prior, the expected divergence of any model containing the uniform distribution is bounded by a constant $1-\gamma $. For the models that we consider this bound is approached as the cardinality of the sample space tends to infinity, if the model dimension remains relatively small. For Dirichlet priors with reasonable concentration parameters the expected values of the divergence behave in a similar way. These results serve as a reference to rank the approximation capabilities of other statistical models.
LA - eng
KW - exponential families; KL divergence; MLE; Dirichlet prior; exponential families; KL divergence; MLE; Dirichlet prior
UR - http://eudml.org/doc/261861
ER -