Average cost Markov control processes with weighted norms: existence of canonical policies

Evgueni Gordienko; Onésimo Hernández-Lerma

Applicationes Mathematicae (1995)

  • Volume: 23, Issue: 2, page 199-218
  • ISSN: 1233-7234

Abstract

top
This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.

How to cite

top

Gordienko, Evgueni, and Hernández-Lerma, Onésimo. "Average cost Markov control processes with weighted norms: existence of canonical policies." Applicationes Mathematicae 23.2 (1995): 199-218. <http://eudml.org/doc/219126>.

@article{Gordienko1995,
abstract = {This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.},
author = {Gordienko, Evgueni, Hernández-Lerma, Onésimo},
journal = {Applicationes Mathematicae},
keywords = {discounted cost; average cost optimality equation; long run average cost; (discrete-time) Markov control processes; average cost optimality inequality; weighted norms; Markov control processes; average cost (in-)equality; discrete; average optimal cost},
language = {eng},
number = {2},
pages = {199-218},
title = {Average cost Markov control processes with weighted norms: existence of canonical policies},
url = {http://eudml.org/doc/219126},
volume = {23},
year = {1995},
}

TY - JOUR
AU - Gordienko, Evgueni
AU - Hernández-Lerma, Onésimo
TI - Average cost Markov control processes with weighted norms: existence of canonical policies
JO - Applicationes Mathematicae
PY - 1995
VL - 23
IS - 2
SP - 199
EP - 218
AB - This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.
LA - eng
KW - discounted cost; average cost optimality equation; long run average cost; (discrete-time) Markov control processes; average cost optimality inequality; weighted norms; Markov control processes; average cost (in-)equality; discrete; average optimal cost
UR - http://eudml.org/doc/219126
ER -

References

top
  1. [1] A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31 (1993), 282-344. Zbl0770.93064
  2. [2] D. P. Bersekas and S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978. 
  3. [3] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979. Zbl0073.34801
  4. [4] E. I. Gordienko, Controlled Markov processes with slowly varying characteristics. The problem of adaptive control. I, Soviet J. Comput. Syst. Sci. 23 (1985), 87-95. Zbl0595.93069
  5. [5] E. I. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration, this volume, 219-237. Zbl0829.93068
  6. [6] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989. 
  7. [7] O. Hernández-Lerma, Average optimality in dynamic programming on Borel spaces-unbounded costs and controls, Systems Control Lett. 17 (1991), 237-242. Zbl0771.90098
  8. [8] O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, ibid. 15 (1990), 349-356. Zbl0723.93080
  9. [9] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation. Zbl0724.93087
  10. [10] O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena, Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28 (1991), 29-46. 
  11. [11] K. Hinderer, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Lecture Notes Oper. Res. 33, Springer, New York, 1970. Zbl0202.18401
  12. [12] N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. I, Theory Probab. Appl. 30 (1985), 247-259. Zbl0657.60088
  13. [13] N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. II, ibid. 30 (1985), 507-515. Zbl0619.60066
  14. [14] N. V. Kartashov, Strongly stable Markov chains, J. Soviet Math. 34 (1986), 1493-1498. Zbl0594.60069
  15. [15] V. K. Malinovskiĭ, Limit theorems for Harris Markov chains, I, Theory Probab. Appl. 31 (1986), 269-285. 
  16. [16] R. Montes-de-Oca and O. Hernández-Lerma, Conditions for average optimality in Markov control processes with unbounded costs and controls, J. Math. Systems Estim. Control 4 (1994), 1-19. Zbl0812.93077
  17. [17] R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear. Zbl0843.93093
  18. [18] E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984. Zbl0551.60066
  19. [19] E. Nummelin and P. Tuominen, Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory, Stochastic Process. Appl. 12 (1982), 187-202. Zbl0484.60056
  20. [20] S. Orey, Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London, 1971. Zbl0295.60054
  21. [21] U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131. Zbl0385.28005
  22. [22] H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971. Zbl0197.03501
  23. [23] M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196. Zbl0316.90080
  24. [24] M. Schäl, Average optimality in dynamic programming with general state space, Math. Oper. Res. 18 (1993), 163-172. Zbl0777.90079
  25. [25] R. Sznajder and J. A. Filar, Some comments on a theorem of Hardy and Littlewood, J. Optim. Theory Appl. 75 (1992), 201-209. 

Citations in EuDML Documents

top
  1. Xiaolong Zou, Xianping Guo, Another set of verifiable conditions for average Markov decision processes with Borel spaces
  2. Fernando Luque-Vásquez, Onésimo Hernández-Lerma, Semi-Markov control models with average costs
  3. Evgueni Gordienko, Onésimo Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration
  4. Evgueni I. Gordienko, Francisco Salem-Silva, Estimates of stability of Markov control processes with unbounded costs
  5. J. Minjárez-Sosa, Nonparametric adaptive control for discrete-time Markov processes with unbounded costs under average criterion
  6. Oscar Vega-Amaya, Fernando Luque-Vásquez, Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times
  7. Onésimo Hernández-Lerma, Oscar Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.