# Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality

Onésimo Hernández-Lerma; Oscar Vega-Amaya

Applicationes Mathematicae (1998)

- Volume: 25, Issue: 2, page 153-178
- ISSN: 1233-7234

## Access Full Article

top## Abstract

top## How to cite

topHernández-Lerma, Onésimo, and Vega-Amaya, Oscar. "Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality." Applicationes Mathematicae 25.2 (1998): 153-178. <http://eudml.org/doc/219198>.

@article{Hernández1998,

abstract = {We consider discrete-time Markov control processes on Borel spaces and infinite-horizon undiscounted cost criteria which are sensitive to the growth rate of finite-horizon costs. These criteria include, at one extreme, the grossly underselective average cost},

author = {Hernández-Lerma, Onésimo, Vega-Amaya, Oscar},

journal = {Applicationes Mathematicae},

keywords = {uniform ergodicity; Lyapunov stability conditions; (discrete-time) Markov control processes; Poisson's equation; undiscounted cost criteria; existence; discrete-time Markov control processes; infinite horizon undiscounted cost criteria},

language = {eng},

number = {2},

pages = {153-178},

title = {Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality},

url = {http://eudml.org/doc/219198},

volume = {25},

year = {1998},

}

TY - JOUR

AU - Hernández-Lerma, Onésimo

AU - Vega-Amaya, Oscar

TI - Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality

JO - Applicationes Mathematicae

PY - 1998

VL - 25

IS - 2

SP - 153

EP - 178

AB - We consider discrete-time Markov control processes on Borel spaces and infinite-horizon undiscounted cost criteria which are sensitive to the growth rate of finite-horizon costs. These criteria include, at one extreme, the grossly underselective average cost

LA - eng

KW - uniform ergodicity; Lyapunov stability conditions; (discrete-time) Markov control processes; Poisson's equation; undiscounted cost criteria; existence; discrete-time Markov control processes; infinite horizon undiscounted cost criteria

UR - http://eudml.org/doc/219198

ER -

## References

top- A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1993), Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31, 282-344. Zbl0770.93064
- R. Bellman (1957), A markovian decision process,z J. Math. Mech. 6, 679-684. Zbl0078.34101
- D. P. Bertsekas and S. E. Shreve (1978), Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York. Zbl0471.93002
- B. W. Brown (1965), On the iterative method of dynamic programming on a finite space discrete time Markov process, Ann. Math. Statist. 33, 719-726.
- D. A. Carlson, A. Haurie and A. Leizarowitz (1991), Infinite Horizon Optimal Control: Deterministic and Stochastic Systems, Springer, New York. Zbl0758.49001
- E. V. Denardo and U. G. Rothblum (1979), Overtaking optimality for Markov decision chains, Math. Oper. Res. 4, 144-152. Zbl0409.90084
- P. K. Dutta (1991), What do discounted optima converge to? A theory of discount rate asymptotics in economic models, J. Econom. Theory 55, 64-94. Zbl0743.90024
- E. B. Dynkin and A. A. Yushkevich (1979), Controlled Markov Processes, Springer, New York. Zbl0073.34801
- A. Ephremides and S. Verdú (1989), Control and optimization methods in communication network problems, IEEE Trans. Automat. Control 34, 930-942. Zbl0709.94667
- E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1994), Controlled Markov processes on the infinite planning horizon: weighted and overtaking criteria, Z. Oper. Res. 39, 131-155. Zbl0809.90128
- J. Flynn (1980), On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76, 202-208. Zbl0438.90100
- D. Gale (1967), On optimal development in a multi-sector economy, Rev. Econom. Stud. 34, 1-19. P. W. Glynn and S. P. Meyn (1996), A Lyapunov bound for solutions of Poisson's equation, Ann. Probab. 24, 916-931.
- E. Gordienko and O. Hernández-Lerma (1995a), Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23, 199-218. Zbl0829.93067
- E. Gordienko and O. Hernández-Lerma (1995b), Average cost Markov control processes with weighted norms: value iteration, ibid., 219-237. Zbl0829.93068
- O. Hernández-Lerma (1989), Adaptive Markov Control Processes, Springer, New York. Zbl0698.90053
- O. Hernández-Lerma, J. C. Hennet and J. B. Lasserre (1991), Average cost Markov decision processes: optimality conditions, J. Math. Anal. Appl. 158, 396-406. Zbl0739.90072
- O. Hernández-Lerma and J. B. Lasserre (1996), Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York. Zbl0840.93001
- O. Hernández-Lerma and J. B. Lasserre (1997), Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math. 47, 125-154. Zbl0872.93080
- O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena (1991), Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28, 29-46. Zbl0717.90087
- O. Hernández-Lerma and M. Muñoz de Ozak (1992), Discrete-time Markov control processes with discounted unbounded cost: optimality criteria, Kybernetika (Prague) 28, 191-212. Zbl0771.93054
- C. J. Himmelberg, T. Parthasarathy and F. S. Van Vleck (1976), Optimal plans for dynamic programming problems, Math. Oper. Res. 1, 390-394. Zbl0368.90134
- A. Leizarowitz (1988), Controlled diffusion processes on infinite horizon with the overtaking criterion, Appl. Math. Optim. 17, 61-78. Zbl0637.60055
- P. Mandl and M. Lausmanová (1991), Two extensions of asymptotic methods in controlled Markov chains, Ann. Oper. Res. 28, 67-79. Zbl0754.60081
- S. P. Meyn (1995), The policy improvement algorithm for Markov decision processes with general state space, preprint, Coordinated Science Laboratory, Univ. of Illinois, Urbana, Ill.
- S. P. Meyn and R. L. Tweedie (1993), Markov Chains and Stochastic Stability, Springer, London. Zbl0925.60001
- R. Montes-de-Oca and O. Hernández-Lerma (1996), Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math. 42, 203-222. Zbl0843.93093
- A. S. Nowak (1992), Stationary overtaking optimal strategies in Markov decision processes with general state space, preprint, Institute of Mathematics, Technical Univ. of Wrocław.
- E. Nummelin (1984), General Irreducible Markov Chains and Non-Negative Operators, Cambridge Univ. Press, Cambridge. Zbl0551.60066
- S. Orey (1971), Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London. Zbl0295.60054
- M. L. Puterman (1994), Markov Decision Processes, Wiley, New York. Zbl0829.90134
- F. P. Ramsey (1928), A mathematical theory of savings, Econom. J. 38, 543-559.
- U. Rieder (1978), Measurable selection theorems for optimization problems, Manuscripta Math. 24, 115-131. Zbl0385.28005
- P. J. Schweitzer (1985), On undiscounted Markovian decision processes with compact action spaces, RAIRO Rech. Opér. 19, 71-86. Zbl0571.90095
- S. Stidham and R. Weber (1993), A survey of Markov decision models for control of networks of queues, Queueing Systems Theory Appl. 13, 291-314. Zbl0772.90082
- O. Vega-Amaya (1996), Overtaking optimality for a class of production-inventory systems, preprint, Departamento de Matemáticas, Universidad de Sonora.
- A. F. Veinott, Jr. (1966), On finding optimal policies in discrete dynamic programming with no discounting, Ann. Math. Statist. 37, 1284-1294. Zbl0149.16301
- C. C. von Weizsäcker (1965), Existence of optimal programs of accumulation for an infinite horizon, Rev. Econom. Stud. 32, 85-104.
- A. A. Yushkevich (1973), On a class of strategies in general Markov decision models, Theory Probab. Appl. 18, 777-779. Zbl0311.90081

## Citations in EuDML Documents

top## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.