Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria

 Access to full text

 Full (PDF)

1. A. Bensoussan, Stochastic control in discrete time and applications to the theory of production, Math. Programm. Study 18 (1982), 43-60. (1982) MR0656937
2. D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N.J. 1987. (1987) Zbl0649.93001MR0896902
3. D. P. Bertsekas, S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York 1978. (1978) Zbl0471.93002MR0511544
4. R.N. Bhattacharya, M. Majumdar, Controlled semi-Markov models - the discounted case, J. Statist. Plann. Inference 21 (1989), 365-381. (1989) Zbl0673.93089MR0995606
5. D. Blackwell, Discounted dynamic programming, Ann. Math. Statist. 36 (1965), 226-235. (1965) Zbl0133.42805MR0173536
6. R.S. Bucy, Stability and positive supermartingales, J. Diff. Eq. 1 (1965), 151-155. (1965) Zbl0203.17505MR0191005
7. R. Cavazos-Cadena, Finite-state approximations for denumerable state discounted Markov decision processes, Appl. Math. Optim. 11, (1986), 1-26. (1986) Zbl0606.90132MR0826849
8. M.H.A. Davis, Martingale methods in stochastic control, Lecture Notes in Control and Inform. Sci. 16 (1979), 85-117. (1979) Zbl0409.93052MR0547467
9. E. B. Dynkin, A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York 1979. (1979) MR0554083
10. O. Hernández-Lerma, Lyapunov criteria for stability of differential equations with Markov parameters, Bol. Soc. Mat. Mexicana 24 (1979), 27-48. (1979) MR0579667
11. O. Hernández-Lerma, Adaptive Markov Control Processes, Springer-Verlag, New York 1989. (1989) MR0995463
12. O. Hernández-Lerma, R. Cavazos-Cadena, Density estimation and adaptive control of Markov processes: average and discounted criteria, Acta Appl. Math. 20 (1990), 285-307. (1990) MR1081591
13. O. Hernández-Lerma, J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, Syst. Control Lett. 15 (1990), 349-356. (1990) MR1078813
14. O. Hernández-Lerma, J. B. Lasserre, Value iteration and rolling plans for Markov control processes with unbounded rewards, J. Math. Anal. Appl. (to appear). MR1224804
15. O Hernández-Lerma, J. B. Lasserre, Error bounds for rolling horizon policies in discrete-time Markov control processes, IEEE Trans. Automat. Control 35 (1990), 1118-1124. (1990) MR1073256
16. O. Hernández-Lerma R. Montes de Oca, R. Cavazos-Cadena, Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28 (1991), 29-46. (1991) MR1105165
17. O. Hernández-Lerma, W. Runggaldier, Monotone approximations for convex stochastic control problems (submitted for publication) 
18. K. Hinderer, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Springer-Verlag, Berlin - Heidelberg - New York 1970. (1970) Zbl0202.18401MR0267890
19. A. Hordijk, H.C. Tijms, A counterexample in discounted dynamic programming, J. Math. Anal. Appl. 39 (1972), 455-457. (1972) Zbl0238.49017MR0307666
20. H.J. Kushner, Optimal discounted stochastic control for diffusion processes, SIAM J. Control 5 (1967), 520-531. (1967) Zbl0178.20003MR0224388
21. S.A. Lippman, On the set of optimal policies in discrete dynamic programming, J. Math. Anal. Appl. 24 (1968), 2,440-445. (1968) Zbl0194.20602MR0231615
22. S.A. Lippman, On dynamic programming with unbounded rewards, Manag. Sci. 21 (1975), 1225-1233. (1975) Zbl0309.90017MR0398535
23. P. Mandl, On the variance in controlled Markov chains, Kybernetika 7 (1971), 1, 1-12. (1971) Zbl0215.25902MR0286178
24. P. Mandl, A connection between controlled Markov chains and martingales, Kybernetika 9 (1973), 4, 237-241. (1973) Zbl0265.60060MR0323427
25. S.P. Meyn, Ergodic theorems for discrete time stochastic systems using a stochastic Lyapunov function, SIAM J. Control Optim. 27 (1989), 1409-1439. (1989) Zbl0681.60067MR1022436
26. U. Rieder, On optimal policies and martingales in dynamic programming, J. Appl. Probab. 13 (1976), 507-518. (1976) Zbl0353.90091MR0421683
27. U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131. (1978) Zbl0385.28005MR0493590
28. M. Schäl, Estimation and control in discounted stochastic dynamic programming, Stochastics 20 (1987), 51-71. (1987) MR0875814
29. J. Wessels, Markov programming by successive approximations with respect to weighted supremum norms, J. Math. Anal. Appl. 58 (1977), 326-335. (1977) Zbl0354.90087MR0441363
30. W. Whitt, Approximations of dynamic programs, I. Math. Oper. Res. 4 (1979), 179-185. (1979) Zbl0408.90082MR0543929

1. Oscar Vega-Amaya, Sample path average optimality of Markov control processes with strictly unbounded cost
2. Yofre H. García, Saul Diaz-Infante, J. Adolfo Minjárez-Sosa, Partially observable queueing systems with controlled service rates under a discounted optimality criterion
3. Onésimo Hernández-Lerma, Oscar Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality