Average cost Markov control processes with weighted norms: existence of canonical policies

 Access to full text

 Full (PDF)

1. [1] A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31 (1993), 282-344. Zbl0770.93064
2. [2] D. P. Bersekas and S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978. 
3. [3] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979. Zbl0073.34801
4. [4] E. I. Gordienko, Controlled Markov processes with slowly varying characteristics. The problem of adaptive control. I, Soviet J. Comput. Syst. Sci. 23 (1985), 87-95. Zbl0595.93069
5. [5] E. I. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration, this volume, 219-237. Zbl0829.93068
6. [6] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989. 
7. [7] O. Hernández-Lerma, Average optimality in dynamic programming on Borel spaces-unbounded costs and controls, Systems Control Lett. 17 (1991), 237-242. Zbl0771.90098
8. [8] O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, ibid. 15 (1990), 349-356. Zbl0723.93080
9. [9] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation. Zbl0724.93087
10. [10] O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena, Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28 (1991), 29-46. 
11. [11] K. Hinderer, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Lecture Notes Oper. Res. 33, Springer, New York, 1970. Zbl0202.18401
12. [12] N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. I, Theory Probab. Appl. 30 (1985), 247-259. Zbl0657.60088
13. [13] N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. II, ibid. 30 (1985), 507-515. Zbl0619.60066
14. [14] N. V. Kartashov, Strongly stable Markov chains, J. Soviet Math. 34 (1986), 1493-1498. Zbl0594.60069
15. [15] V. K. Malinovskiĭ, Limit theorems for Harris Markov chains, I, Theory Probab. Appl. 31 (1986), 269-285. 
16. [16] R. Montes-de-Oca and O. Hernández-Lerma, Conditions for average optimality in Markov control processes with unbounded costs and controls, J. Math. Systems Estim. Control 4 (1994), 1-19. Zbl0812.93077
17. [17] R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear. Zbl0843.93093
18. [18] E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984. Zbl0551.60066
19. [19] E. Nummelin and P. Tuominen, Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory, Stochastic Process. Appl. 12 (1982), 187-202. Zbl0484.60056
20. [20] S. Orey, Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London, 1971. Zbl0295.60054
21. [21] U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131. Zbl0385.28005
22. [22] H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971. Zbl0197.03501
23. [23] M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196. Zbl0316.90080
24. [24] M. Schäl, Average optimality in dynamic programming with general state space, Math. Oper. Res. 18 (1993), 163-172. Zbl0777.90079
25. [25] R. Sznajder and J. A. Filar, Some comments on a theorem of Hardy and Littlewood, J. Optim. Theory Appl. 75 (1992), 201-209. 

1. Xiaolong Zou, Xianping Guo, Another set of verifiable conditions for average Markov decision processes with Borel spaces
2. Fernando Luque-Vásquez, Onésimo Hernández-Lerma, Semi-Markov control models with average costs
3. Evgueni Gordienko, Onésimo Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration
4. Evgueni I. Gordienko, Francisco Salem-Silva, Estimates of stability of Markov control processes with unbounded costs
5. J. Minjárez-Sosa, Nonparametric adaptive control for discrete-time Markov processes with unbounded costs under average criterion
6. Fernando Luque-Vásquez, J. Adolfo Minjárez-Sosa, Empirical approximation in Markov games under unbounded payoff: discounted and average criteria
7. Oscar Vega-Amaya, Fernando Luque-Vásquez, Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times
8. Onésimo Hernández-Lerma, Oscar Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality