Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality

 Access to full text

 Full (PDF)

1. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1993), Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31, 282-344. Zbl0770.93064
2. R. Bellman (1957), A markovian decision process,z J. Math. Mech. 6, 679-684. Zbl0078.34101
3. D. P. Bertsekas and S. E. Shreve (1978), Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York. Zbl0471.93002
4. B. W. Brown (1965), On the iterative method of dynamic programming on a finite space discrete time Markov process, Ann. Math. Statist. 33, 719-726. 
5. D. A. Carlson, A. Haurie and A. Leizarowitz (1991), Infinite Horizon Optimal Control: Deterministic and Stochastic Systems, Springer, New York. Zbl0758.49001
6. E. V. Denardo and U. G. Rothblum (1979), Overtaking optimality for Markov decision chains, Math. Oper. Res. 4, 144-152. Zbl0409.90084
7. P. K. Dutta (1991), What do discounted optima converge to? A theory of discount rate asymptotics in economic models, J. Econom. Theory 55, 64-94. Zbl0743.90024
8. E. B. Dynkin and A. A. Yushkevich (1979), Controlled Markov Processes, Springer, New York. Zbl0073.34801
9. A. Ephremides and S. Verdú (1989), Control and optimization methods in communication network problems, IEEE Trans. Automat. Control 34, 930-942. Zbl0709.94667
10. E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1994), Controlled Markov processes on the infinite planning horizon: weighted and overtaking criteria, Z. Oper. Res. 39, 131-155. Zbl0809.90128
11. J. Flynn (1980), On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76, 202-208. Zbl0438.90100
12. D. Gale (1967), On optimal development in a multi-sector economy, Rev. Econom. Stud. 34, 1-19. P. W. Glynn and S. P. Meyn (1996), A Lyapunov bound for solutions of Poisson's equation, Ann. Probab. 24, 916-931. 
13. E. Gordienko and O. Hernández-Lerma (1995a), Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23, 199-218. Zbl0829.93067
14. E. Gordienko and O. Hernández-Lerma (1995b), Average cost Markov control processes with weighted norms: value iteration, ibid., 219-237. Zbl0829.93068
15. O. Hernández-Lerma (1989), Adaptive Markov Control Processes, Springer, New York. Zbl0698.90053
16. O. Hernández-Lerma, J. C. Hennet and J. B. Lasserre (1991), Average cost Markov decision processes: optimality conditions, J. Math. Anal. Appl. 158, 396-406. Zbl0739.90072
17. O. Hernández-Lerma and J. B. Lasserre (1996), Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York. Zbl0840.93001
18. O. Hernández-Lerma and J. B. Lasserre (1997), Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math. 47, 125-154. Zbl0872.93080
19. O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena (1991), Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28, 29-46. Zbl0717.90087
20. O. Hernández-Lerma and M. Muñoz de Ozak (1992), Discrete-time Markov control processes with discounted unbounded cost: optimality criteria, Kybernetika (Prague) 28, 191-212. Zbl0771.93054
21. C. J. Himmelberg, T. Parthasarathy and F. S. Van Vleck (1976), Optimal plans for dynamic programming problems, Math. Oper. Res. 1, 390-394. Zbl0368.90134
22. A. Leizarowitz (1988), Controlled diffusion processes on infinite horizon with the overtaking criterion, Appl. Math. Optim. 17, 61-78. Zbl0637.60055
23. P. Mandl and M. Lausmanová (1991), Two extensions of asymptotic methods in controlled Markov chains, Ann. Oper. Res. 28, 67-79. Zbl0754.60081
24. S. P. Meyn (1995), The policy improvement algorithm for Markov decision processes with general state space, preprint, Coordinated Science Laboratory, Univ. of Illinois, Urbana, Ill. 
25. S. P. Meyn and R. L. Tweedie (1993), Markov Chains and Stochastic Stability, Springer, London. Zbl0925.60001
26. R. Montes-de-Oca and O. Hernández-Lerma (1996), Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math. 42, 203-222. Zbl0843.93093
27. A. S. Nowak (1992), Stationary overtaking optimal strategies in Markov decision processes with general state space, preprint, Institute of Mathematics, Technical Univ. of Wrocław. 
28. E. Nummelin (1984), General Irreducible Markov Chains and Non-Negative Operators, Cambridge Univ. Press, Cambridge. Zbl0551.60066
29. S. Orey (1971), Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London. Zbl0295.60054
30. M. L. Puterman (1994), Markov Decision Processes, Wiley, New York. Zbl0829.90134
31. F. P. Ramsey (1928), A mathematical theory of savings, Econom. J. 38, 543-559. 
32. U. Rieder (1978), Measurable selection theorems for optimization problems, Manuscripta Math. 24, 115-131. Zbl0385.28005
33. P. J. Schweitzer (1985), On undiscounted Markovian decision processes with compact action spaces, RAIRO Rech. Opér. 19, 71-86. Zbl0571.90095
34. S. Stidham and R. Weber (1993), A survey of Markov decision models for control of networks of queues, Queueing Systems Theory Appl. 13, 291-314. Zbl0772.90082
35. O. Vega-Amaya (1996), Overtaking optimality for a class of production-inventory systems, preprint, Departamento de Matemáticas, Universidad de Sonora. 
36. A. F. Veinott, Jr. (1966), On finding optimal policies in discrete dynamic programming with no discounting, Ann. Math. Statist. 37, 1284-1294. Zbl0149.16301
37. C. C. von Weizsäcker (1965), Existence of optimal programs of accumulation for an infinite horizon, Rev. Econom. Stud. 32, 85-104. 
38. A. A. Yushkevich (1973), On a class of strategies in general Markov decision models, Theory Probab. Appl. 18, 777-779. Zbl0311.90081

1. Oscar Vega-Amaya, Fernando Luque-Vásquez, Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times