Stochastic control optimal in the Kullback sense

Jan Šindelář; Igor Vajda; Miroslav Kárný

Kybernetika (2008)

  • Volume: 44, Issue: 1, page 53-60
  • ISSN: 0023-5954

Abstract

top
The paper solves the problem of minimization of the Kullback divergence between a partially known and a completely known probability distribution. It considers two probability distributions of a random vector ( u 1 , x 1 , ... , u T , x T ) on a sample space of 2 T dimensions. One of the distributions is known, the other is known only partially. Namely, only the conditional probability distributions of x τ given u 1 , x 1 , ... , u τ - 1 , x τ - 1 , u τ are known for τ = 1 , ... , T . Our objective is to determine the remaining conditional probability distributions of u τ given u 1 , x 1 , ... , u τ - 1 , x τ - 1 such that the Kullback divergence of the partially known distribution with respect to the completely known distribution is minimal. Explicit solution of this problem has been found previously for Markovian systems in Karný [Karny:96a]. The general solution is given in this paper.

How to cite

top

Šindelář, Jan, Vajda, Igor, and Kárný, Miroslav. "Stochastic control optimal in the Kullback sense." Kybernetika 44.1 (2008): 53-60. <http://eudml.org/doc/33912>.

@article{Šindelář2008,
abstract = {The paper solves the problem of minimization of the Kullback divergence between a partially known and a completely known probability distribution. It considers two probability distributions of a random vector $(u_1, x_1, \ldots , u_T, x_T )$ on a sample space of $2T$ dimensions. One of the distributions is known, the other is known only partially. Namely, only the conditional probability distributions of $x_\tau $ given $u_1, x_1, \ldots , u_\{\tau -1\}, x_\{\tau -1\}, u_\{\tau \}$ are known for $\tau = 1, \ldots , T$. Our objective is to determine the remaining conditional probability distributions of $u_\tau $ given $u_1, x_1, \ldots , u_\{\tau -1\}, x_\{\tau -1\}$ such that the Kullback divergence of the partially known distribution with respect to the completely known distribution is minimal. Explicit solution of this problem has been found previously for Markovian systems in Karný [Karny:96a]. The general solution is given in this paper.},
author = {Šindelář, Jan, Vajda, Igor, Kárný, Miroslav},
journal = {Kybernetika},
keywords = {Kullback divergence; minimization; stochastic controller; Kullback divergence; minimization; stochastic controller},
language = {eng},
number = {1},
pages = {53-60},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Stochastic control optimal in the Kullback sense},
url = {http://eudml.org/doc/33912},
volume = {44},
year = {2008},
}

TY - JOUR
AU - Šindelář, Jan
AU - Vajda, Igor
AU - Kárný, Miroslav
TI - Stochastic control optimal in the Kullback sense
JO - Kybernetika
PY - 2008
PB - Institute of Information Theory and Automation AS CR
VL - 44
IS - 1
SP - 53
EP - 60
AB - The paper solves the problem of minimization of the Kullback divergence between a partially known and a completely known probability distribution. It considers two probability distributions of a random vector $(u_1, x_1, \ldots , u_T, x_T )$ on a sample space of $2T$ dimensions. One of the distributions is known, the other is known only partially. Namely, only the conditional probability distributions of $x_\tau $ given $u_1, x_1, \ldots , u_{\tau -1}, x_{\tau -1}, u_{\tau }$ are known for $\tau = 1, \ldots , T$. Our objective is to determine the remaining conditional probability distributions of $u_\tau $ given $u_1, x_1, \ldots , u_{\tau -1}, x_{\tau -1}$ such that the Kullback divergence of the partially known distribution with respect to the completely known distribution is minimal. Explicit solution of this problem has been found previously for Markovian systems in Karný [Karny:96a]. The general solution is given in this paper.
LA - eng
KW - Kullback divergence; minimization; stochastic controller; Kullback divergence; minimization; stochastic controller
UR - http://eudml.org/doc/33912
ER -

References

top
  1. Aoki M., Optimization of Stochastic Systems: Topics in Discrete-Time Systems, Academic Press, New York – London 1967 Zbl0168.15802MR0234749
  2. Åström K. J., Introduction to Stochastic Control, Academic Press, New York – San Francisco – London 1970 Zbl1191.93141
  3. Bertsekas D. P., Dynamic Programming and Stochastic Control, Second edition. Athena Scientific, Belmont, Mass. 2000 Zbl0549.93064MR2182753
  4. Clark D., Advances in Model-Based Predictive Control, Oxford University Press, Oxford 1994 
  5. Cover T. M., Thomas J. A., Elements of Information Theory, Second edition. Wiley-Interscience, New York 2006 Zbl1140.94001MR2239987
  6. Kárný M., Towards fully probabilistic control design, Automatica 32 (1996), 12, 1719–1722 (1996) Zbl0868.93022MR1427142
  7. Kulhavý R., A Kullback–Leibler distance approach to system identification, In: Preprints of the IFAC Symposium on Adaptive Systems in Control and Signal Processing (C. Bányász, ed.), Budapest 1995, pp. 55–66 (1995) 
  8. Kullback S., Information Theory and Statistics, Wiley, New York and Chapman & Hall, London 1967 Zbl0897.62003MR0103557
  9. Kullback S., Leibler R., On information and sufficiency, Ann. Math. Statist. 22 (1951), 79–87 (1951) Zbl0042.38403MR0039968
  10. Kumar P. R., Varaiya P., Stochastic Systems: Estimation, Identification and Adaptive Control, Prentice Hall, Englewood Cliffs, N. J. 1986 Zbl0706.93057
  11. Kushner H., Introduction to Stochastic Control, Holt, Rinehard and Winston, New York 1971 Zbl0293.93018MR0280248
  12. Martin J. J., Bayesian Decision Problems and Markov Chains, Wiley, New York 1967 Zbl0164.50102MR0221709
  13. Meditch J. S., Stochastic Optimal Linear Estimation and Control, Mc. Graw Hill, New York 1969 Zbl0269.93061
  14. Vajda I., Theory of Statistical Inference and Information, Mathematical and statistical methods. Kluwer Academic Publishers, Dordrecht, Boston – London 1989 Zbl0711.62002

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.