Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria

Onésimo Hernández-Lerma; Myriam Muñoz de Ozak

Kybernetika (1992)

  • Volume: 28, Issue: 3, page 191-212
  • ISSN: 0023-5954

How to cite

top

Hernández-Lerma, Onésimo, and Muñoz de Ozak, Myriam. "Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria." Kybernetika 28.3 (1992): 191-212. <http://eudml.org/doc/27742>.

@article{Hernández1992,
author = {Hernández-Lerma, Onésimo, Muñoz de Ozak, Myriam},
journal = {Kybernetika},
keywords = {discrete-time Markov control processes; Borel state; optimal cost function; Bellman's principle of optimality},
language = {eng},
number = {3},
pages = {191-212},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria},
url = {http://eudml.org/doc/27742},
volume = {28},
year = {1992},
}

TY - JOUR
AU - Hernández-Lerma, Onésimo
AU - Muñoz de Ozak, Myriam
TI - Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria
JO - Kybernetika
PY - 1992
PB - Institute of Information Theory and Automation AS CR
VL - 28
IS - 3
SP - 191
EP - 212
LA - eng
KW - discrete-time Markov control processes; Borel state; optimal cost function; Bellman's principle of optimality
UR - http://eudml.org/doc/27742
ER -

References

top
  1. A. Bensoussan, Stochastic control in discrete time and applications to the theory of production, Math. Programm. Study 18 (1982), 43-60. (1982) MR0656937
  2. D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N.J. 1987. (1987) Zbl0649.93001MR0896902
  3. D. P. Bertsekas, S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York 1978. (1978) Zbl0471.93002MR0511544
  4. R.N. Bhattacharya, M. Majumdar, Controlled semi-Markov models - the discounted case, J. Statist. Plann. Inference 21 (1989), 365-381. (1989) Zbl0673.93089MR0995606
  5. D. Blackwell, Discounted dynamic programming, Ann. Math. Statist. 36 (1965), 226-235. (1965) Zbl0133.42805MR0173536
  6. R.S. Bucy, Stability and positive supermartingales, J. Diff. Eq. 1 (1965), 151-155. (1965) Zbl0203.17505MR0191005
  7. R. Cavazos-Cadena, Finite-state approximations for denumerable state discounted Markov decision processes, Appl. Math. Optim. 11, (1986), 1-26. (1986) Zbl0606.90132MR0826849
  8. M.H.A. Davis, Martingale methods in stochastic control, Lecture Notes in Control and Inform. Sci. 16 (1979), 85-117. (1979) Zbl0409.93052MR0547467
  9. E. B. Dynkin, A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York 1979. (1979) MR0554083
  10. O. Hernández-Lerma, Lyapunov criteria for stability of differential equations with Markov parameters, Bol. Soc. Mat. Mexicana 24 (1979), 27-48. (1979) MR0579667
  11. O. Hernández-Lerma, Adaptive Markov Control Processes, Springer-Verlag, New York 1989. (1989) MR0995463
  12. O. Hernández-Lerma, R. Cavazos-Cadena, Density estimation and adaptive control of Markov processes: average and discounted criteria, Acta Appl. Math. 20 (1990), 285-307. (1990) MR1081591
  13. O. Hernández-Lerma, J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, Syst. Control Lett. 15 (1990), 349-356. (1990) MR1078813
  14. O. Hernández-Lerma, J. B. Lasserre, Value iteration and rolling plans for Markov control processes with unbounded rewards, J. Math. Anal. Appl. (to appear). MR1224804
  15. O Hernández-Lerma, J. B. Lasserre, Error bounds for rolling horizon policies in discrete-time Markov control processes, IEEE Trans. Automat. Control 35 (1990), 1118-1124. (1990) MR1073256
  16. O. Hernández-Lerma R. Montes de Oca, R. Cavazos-Cadena, Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28 (1991), 29-46. (1991) MR1105165
  17. O. Hernández-Lerma, W. Runggaldier, Monotone approximations for convex stochastic control problems (submitted for publication) 
  18. K. Hinderer, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Springer-Verlag, Berlin - Heidelberg - New York 1970. (1970) Zbl0202.18401MR0267890
  19. A. Hordijk, H.C. Tijms, A counterexample in discounted dynamic programming, J. Math. Anal. Appl. 39 (1972), 455-457. (1972) Zbl0238.49017MR0307666
  20. H.J. Kushner, Optimal discounted stochastic control for diffusion processes, SIAM J. Control 5 (1967), 520-531. (1967) Zbl0178.20003MR0224388
  21. S.A. Lippman, On the set of optimal policies in discrete dynamic programming, J. Math. Anal. Appl. 24 (1968), 2,440-445. (1968) Zbl0194.20602MR0231615
  22. S.A. Lippman, On dynamic programming with unbounded rewards, Manag. Sci. 21 (1975), 1225-1233. (1975) Zbl0309.90017MR0398535
  23. P. Mandl, On the variance in controlled Markov chains, Kybernetika 7 (1971), 1, 1-12. (1971) Zbl0215.25902MR0286178
  24. P. Mandl, A connection between controlled Markov chains and martingales, Kybernetika 9 (1973), 4, 237-241. (1973) Zbl0265.60060MR0323427
  25. S.P. Meyn, Ergodic theorems for discrete time stochastic systems using a stochastic Lyapunov function, SIAM J. Control Optim. 27 (1989), 1409-1439. (1989) Zbl0681.60067MR1022436
  26. U. Rieder, On optimal policies and martingales in dynamic programming, J. Appl. Probab. 13 (1976), 507-518. (1976) Zbl0353.90091MR0421683
  27. U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131. (1978) Zbl0385.28005MR0493590
  28. M. Schäl, Estimation and control in discounted stochastic dynamic programming, Stochastics 20 (1987), 51-71. (1987) MR0875814
  29. J. Wessels, Markov programming by successive approximations with respect to weighted supremum norms, J. Math. Anal. Appl. 58 (1977), 326-335. (1977) Zbl0354.90087MR0441363
  30. W. Whitt, Approximations of dynamic programs, I. Math. Oper. Res. 4 (1979), 179-185. (1979) Zbl0408.90082MR0543929

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.