Optimally approximating exponential families
Kybernetika (2013)
- Volume: 49, Issue: 2, page 199-215
- ISSN: 0023-5954
Access Full Article
topAbstract
topHow to cite
topRauh, Johannes. "Optimally approximating exponential families." Kybernetika 49.2 (2013): 199-215. <http://eudml.org/doc/260703>.
@article{Rauh2013,
abstract = {This article studies exponential families $\mathcal \{E\}$ on finite sets such that the information divergence $D(P\Vert \mathcal \{E\})$ of an arbitrary probability distribution from $\mathcal \{E\}$ is bounded by some constant $D>0$. A particular class of low-dimensional exponential families that have low values of $D$ can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. The case where $D=\log (2)$ is studied in detail. This case is special, because if $D<\log (2)$, then $\mathcal \{E\}$ contains all probability measures with full support.},
author = {Rauh, Johannes},
journal = {Kybernetika},
keywords = {exponential family; information divergence; exponential family; information divergence},
language = {eng},
number = {2},
pages = {199-215},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Optimally approximating exponential families},
url = {http://eudml.org/doc/260703},
volume = {49},
year = {2013},
}
TY - JOUR
AU - Rauh, Johannes
TI - Optimally approximating exponential families
JO - Kybernetika
PY - 2013
PB - Institute of Information Theory and Automation AS CR
VL - 49
IS - 2
SP - 199
EP - 215
AB - This article studies exponential families $\mathcal {E}$ on finite sets such that the information divergence $D(P\Vert \mathcal {E})$ of an arbitrary probability distribution from $\mathcal {E}$ is bounded by some constant $D>0$. A particular class of low-dimensional exponential families that have low values of $D$ can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. The case where $D=\log (2)$ is studied in detail. This case is special, because if $D<\log (2)$, then $\mathcal {E}$ contains all probability measures with full support.
LA - eng
KW - exponential family; information divergence; exponential family; information divergence
UR - http://eudml.org/doc/260703
ER -
References
top- Ay, N., 10.1214/aop/1020107773, Ann. Probab. 30 (2002), 416-436. Zbl1010.62007MR1894113DOI10.1214/aop/1020107773
- Ay, N., 10.1162/089976602760805368, Neural Computat. 14 (2002), 2959-2980. Zbl1079.68582DOI10.1162/089976602760805368
- Brown, L., Fundamentals of Statistical Exponential Families: With Applications in Statistical Decision Theory., Institute of Mathematical Statistics, Hayworth 1986. Zbl0685.62002MR0882001
- Cover, T., Thomas, J., Elements of Information Theory. First edition., Wiley, 1991. MR1122806
- Csiszár, I., Shields, P., Information Theory and Statistics: A Tutorial. First edition., Foundations and Trends in Communications and Information Theory. Now Publishers, 2004.
- Csiszár, I., Matúš, F., 10.1007/s00440-007-0084-z, Probab. Theory Rel. Fields 141 (2008), 213-246. MR2372970DOI10.1007/s00440-007-0084-z
- Pietra, S. Della, Pietra, V. Della, Lafferty, J., 10.1109/34.588021, IEEE Trans. Pattern Analysis and Machine Intelligence 19 (1997), 380-393. DOI10.1109/34.588021
- Drton, M., Sturmfels, B., Sullivant, S., Lectures on algebraic statistics., In: Oberwolfach Seminars 39, Birkhäuser, Basel 2009. Zbl1166.13001MR2723140
- Geiger, D., Meek, C., Sturmfels, B., 10.1214/009053606000000263, Ann. Statist. 34 (2006), 5, 1463-1492. Zbl1104.60007MR2278364DOI10.1214/009053606000000263
- Jaynes, E. T., 10.1103/PhysRev.106.620, Phys. Rev. 106 (1957), 4, 620-630. Zbl0084.43701MR0087305DOI10.1103/PhysRev.106.620
- Juríček, J., Maximization of information divergence from multinomial distributions., Acta Univ. Carolin. 52 (2011), 1, 27-35. MR2808291
- Lauritzen, S. L., Graphical Models. First edition., Oxford Statistical Science Series, Oxford University Press, 1996. MR1419991
- Linsker, R., 10.1109/2.36, IEEE Computer 21 (1988), 105-117. DOI10.1109/2.36
- Matúš, F., Ay, N., On maximization of the information divergence from an exponential family., In: Proc. WUPES'03, University of Economics, Prague 2003, pp. 199-204.
- Matúš, F., Rauh, J., Maximization of the information divergence from an exponential family and criticality., In: 2011 IEEE International Symposium on Information Theory Proceedings (ISIT2011), 2011.
- Montúfar, G., Rauh, J., Ay, N., Expressive power and approximation errors of Restricted Boltzmann Machines., In: NIPS, 2011.
- Oxley, J., Matroid Theory. First edition., Oxford University Press, New York 1992. MR1207587
- Rauh, J., Finding the Maximizers of the Information Divergence from an Exponential Family., Ph.D. Dissertation, Universität Leipzig, 2011. MR2817016
- Rauh, J., 10.1109/TIT.2011.2136230, IEEE Trans. Inform. Theory 57 (2011), 6, 3236-3247. MR2817016DOI10.1109/TIT.2011.2136230
- Rauh, J., Kahle, T., Ay, N., 10.1016/j.ijar.2011.01.013, Internat. J. Approx. Reasoning 52 (2011), 5, 613-626. MR2787021DOI10.1016/j.ijar.2011.01.013
- Zhu, S. C., Wu, Y. N., Mumford, D., 10.1162/neco.1997.9.8.1627, Neural Computation 9 (1997), 1627-1660. DOI10.1162/neco.1997.9.8.1627
Citations in EuDML Documents
topNotesEmbed ?
topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.