The risk-sensitive Poisson equation for a communicating Markov chain on a denumerable state space

Rolando Cavazos-Cadena

Kybernetika (2009)

  • Volume: 45, Issue: 5, page 716-736
  • ISSN: 0023-5954

Abstract

top
This work concerns a discrete-time Markov chain with time-invariant transition mechanism and denumerable state space, which is endowed with a nonnegative cost function with finite support. The performance of the chain is measured by the (long-run) risk-sensitive average cost and, assuming that the state space is communicating, the existence of a solution to the risk-sensitive Poisson equation is established, a result that holds even for transient chains. Also, a sufficient criterion ensuring that the functional part of a solution is uniquely determined up to an additive constant is provided, and an example is given to show that the uniqueness result may fail when that criterion is not satisfied.

How to cite

top

Cavazos-Cadena, Rolando. "The risk-sensitive Poisson equation for a communicating Markov chain on a denumerable state space." Kybernetika 45.5 (2009): 716-736. <http://eudml.org/doc/37699>.

@article{Cavazos2009,
abstract = {This work concerns a discrete-time Markov chain with time-invariant transition mechanism and denumerable state space, which is endowed with a nonnegative cost function with finite support. The performance of the chain is measured by the (long-run) risk-sensitive average cost and, assuming that the state space is communicating, the existence of a solution to the risk-sensitive Poisson equation is established, a result that holds even for transient chains. Also, a sufficient criterion ensuring that the functional part of a solution is uniquely determined up to an additive constant is provided, and an example is given to show that the uniqueness result may fail when that criterion is not satisfied.},
author = {Cavazos-Cadena, Rolando},
journal = {Kybernetika},
keywords = {possibly transient Markov chains; discounted approach; first return time; uniqueness of solutions to the multiplicative Poisson equation; possibly transient Markov chains; discounted approach; first return time; uniqueness of solutions to the multiplicative Poisson equation},
language = {eng},
number = {5},
pages = {716-736},
publisher = {Institute of Information Theory and Automation AS CR},
title = {The risk-sensitive Poisson equation for a communicating Markov chain on a denumerable state space},
url = {http://eudml.org/doc/37699},
volume = {45},
year = {2009},
}

TY - JOUR
AU - Cavazos-Cadena, Rolando
TI - The risk-sensitive Poisson equation for a communicating Markov chain on a denumerable state space
JO - Kybernetika
PY - 2009
PB - Institute of Information Theory and Automation AS CR
VL - 45
IS - 5
SP - 716
EP - 736
AB - This work concerns a discrete-time Markov chain with time-invariant transition mechanism and denumerable state space, which is endowed with a nonnegative cost function with finite support. The performance of the chain is measured by the (long-run) risk-sensitive average cost and, assuming that the state space is communicating, the existence of a solution to the risk-sensitive Poisson equation is established, a result that holds even for transient chains. Also, a sufficient criterion ensuring that the functional part of a solution is uniquely determined up to an additive constant is provided, and an example is given to show that the uniqueness result may fail when that criterion is not satisfied.
LA - eng
KW - possibly transient Markov chains; discounted approach; first return time; uniqueness of solutions to the multiplicative Poisson equation; possibly transient Markov chains; discounted approach; first return time; uniqueness of solutions to the multiplicative Poisson equation
UR - http://eudml.org/doc/37699
ER -

References

top
  1. Discrete-time controlled Markov processes with average cost criteria: a survey, SIAM J. Control Optim. 31 (1993), 282–334. MR1205981
  2. Controlled Markov chains with a risk-sensitive criteria: some counterexamples In: Proc, 37th IEEE Conference on Decision and Control, Tempa 1998, pp. 1853–1858. 
  3. Controlled Markov chains with risk-sensitive criteria: average cost, optimality equations and optimal solutions, Math. Methods Oper. Res. 43 (1999), 121–139. MR1687362
  4. Risk-sensitive control in communicating average Markov decision chains, In: Modelling Uncertainty: An examination of Stochastic Theory, Methods and Applications (M. Dror, P. L’Ecuyer, and F. Szidarovsky, eds.), Kluwer, Boston 2002, pp. 525–544. 
  5. Solution to the risk-sensitive average cost optimality equation in communicating Markov decision chains with finite state space: An alternative approach, Math. Methods Oper. Res. 56 (2003), 473–479. MR1953028
  6. Solution to the risk-sensitive average cost optimality equation in a class of Markov decision processes with finite state space, Math. Methods Oper. Res. 57 (2003), 263–285. MR1973378
  7. A characterization of the optimal risk-sensitive average cost in finite controlled Markov chains, Ann. Appl. Probab. 15 2005, 175–212. MR2115041
  8. Necessary and sufficient conditions for a solution to the risk-sensitive Poisson equation on a finite state space, Systems Control Lett. 58 (2009), 254–258. MR2510639
  9. Risk-sensitive control of discrete time Markov processes with infinite horizon, SIAM J. Control Optim. 38 (1999), 61–78. MR1740607
  10. Risk-sensitive control on an infinite horizon, SIAM J. Control Optim. 33 (1995), 1881–1915. MR1358100
  11. Risk-sensitive control of Markov processes in countable state space, Systems Control Lett. 29 (1996), 147–155. MR1422212
  12. [unknown], O. Hernández-Lerma: Adaptive Markov Control Processes Springer, New York 1988. MR0995463
  13. Risk-sensitive Markov decision processes, Management Sci. 18 (1972), 356–369. MR0292497
  14. Optimal stochastic linear systems with exponential performance criteria and their relation to stochastic differential games, IEEE Trans. Automat. Control 18 (1973), 124–131. MR0441523
  15. Markov decison processes with a new optimality criterion: discrete time, Ann. Statist. 1 (1973), 496–505. MR0378839
  16. A utility criterion for Markov decision processes, Management Sci. 23 (1976), 43–49. Zbl0337.90053MR0439037
  17. Average optimality for risk sensitive control with general state space, Ann. Appl. Probab. 17 (2007), 654–675. MR2308338
  18. Probability Theory I, Springer, New York 1980. MR0651017
  19. Markov Decision Processes, Wiley, New York 1994. Zbl1184.90170MR1270015
  20. Nonnegative Matrices, Springer, New York 1980. 
  21. Growth rates and average optimality in risk-sensitive Markov decision chains, Kybernetika 44 (2008), 205–226. MR2428220
  22. Risk-sensitive average optimality in Markov decision chains Raul, In: Oper. Res. Proc. 2007 (Selected Papers of the Internat. Conference on Operations Research 2007, Saarbruecken, J. Kalcsics and S. Nickel, eds.), Springer-Verlag, Berlin – Heidelberg 2008, pp. 69–74. 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.