Semi-Markov control models with average costs

Fernando Luque-Vásquez; Onésimo Hernández-Lerma

Applicationes Mathematicae (1999)

  • Volume: 26, Issue: 3, page 315-331
  • ISSN: 1233-7234

Abstract

top
This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.

How to cite

top

Luque-Vásquez, Fernando, and Hernández-Lerma, Onésimo. "Semi-Markov control models with average costs." Applicationes Mathematicae 26.3 (1999): 315-331. <http://eudml.org/doc/219242>.

@article{Luque1999,
abstract = {This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.},
author = {Luque-Vásquez, Fernando, Hernández-Lerma, Onésimo},
journal = {Applicationes Mathematicae},
keywords = {average cost; replacement models; semi-Markov control models; policy iteration (or Howard's algorithm)},
language = {eng},
number = {3},
pages = {315-331},
title = {Semi-Markov control models with average costs},
url = {http://eudml.org/doc/219242},
volume = {26},
year = {1999},
}

TY - JOUR
AU - Luque-Vásquez, Fernando
AU - Hernández-Lerma, Onésimo
TI - Semi-Markov control models with average costs
JO - Applicationes Mathematicae
PY - 1999
VL - 26
IS - 3
SP - 315
EP - 331
AB - This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.
LA - eng
KW - average cost; replacement models; semi-Markov control models; policy iteration (or Howard's algorithm)
UR - http://eudml.org/doc/219242
ER -

References

top
  1. [1] R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972. 
  2. [2] R. N. Bhattacharya and M. Majumdar, Controlled semi-Markov models under long-run average rewards, J. Statist. Plann. Inference 22 (1989), 223-242. Zbl0683.49003
  3. [3] A. Federgruen, A. Hordijk and H. C. Tijms, Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion, Stochastic Process. Appl. 9 (1979), 222-235. Zbl0422.90084
  4. [4] A. Federgruen, P. J. Schweitzer and H. C. Tijms, Denumerable undiscounted semi-Markov decision processes with unbounded rewards, Math. Oper. Res. 8 (1983), 298-313. Zbl0513.90085
  5. [5] A. Federgruen and H. C. Tijms, The optimality equation in average cost denumerable state semi-Markov decision problems. Recurrence conditions and algorithms, J. Appl. Probab. 15 (1978), 356-373. Zbl0386.90060
  6. [6] P. W. Glynn and S. P. Meyn, A Lyapounov bound for solutions of Poisson's equations, Ann. Probab. 24 (1996), 916-931. Zbl0863.60063
  7. [7] E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23 (1995), 199-218. Zbl0829.93067
  8. [8] U. G. Haussmann, On the optimal long-run control of Markov renewal processes, J. Math. Anal. Appl. 36 (1971), 123-140. Zbl0195.16301
  9. [9] O. Hernández-Lerma and J. B. Lasserre, Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math. 47 (1997), 125-154. Zbl0872.93080
  10. [10] O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes, Springer, New York, 1999 (in press). Zbl0928.93002
  11. [11] M. Kurano, Semi-Markov decision processes and their applications in the replacement models, J. Oper. Res. Soc. Japan 28 (1985), 18-29. Zbl0564.90090
  12. [12] S. A. Lippman, Semi-Markov decision processes with unbounded rewards, Management Sci. 19 (1973), 717-731. Zbl0259.60044
  13. [13] S. A. Lippman, On dynamic programming with unbounded rewards, ibid. 21 (1975), 1225-1233. Zbl0309.90017
  14. [14] S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer, London, 1993. Zbl0925.60001
  15. [15] E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge Univ. Press, Cambridge, 1984. Zbl0551.60066
  16. [16] M. L. Puterman, Markov Decision Processes. Discrete Stochastic Dynamic Programming, Wiley, New York, 1994. 
  17. [17] U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131. Zbl0385.28005
  18. [18] S. M. Ross, Applied Probability Models with Optimization Applications, Holden-Day, San Francisco, 1970. Zbl0213.19101
  19. [19] S. M. Ross, Average cost semi-Markov decision processes, J. Appl. Probab. 7 (1970), 649-656. Zbl0204.51704
  20. [20] M. Schäl, On the second optimality equation for semi-Markov decision models, Math. Oper. Res. 17 (1992), 470-486. Zbl0773.90091
  21. [21] P. J. Schweitzer, Iterative solution of the functional equations of undiscounted Markov renewal programming, J. Math. Anal. Appl. 34 (1971), 495-501. Zbl0218.90070
  22. [22] L. I. Sennott, Average cost semi-Markov decision processes and the control of queueing systems, Probab. Engrg. Inform. Sci. 3 (1989), 247-272. Zbl1134.60408
  23. [23] H. C. Tijms, Stochastic Models: An Algorithmic Approach, Wiley, Chichester, 1994. 
  24. [24] O. Vega-Amaya, Markov control processes in Borel spaces: undiscounted criteria, doctoral thesis, UAM-Iztapalapa, México, 1998 (in Spanish). 

NotesEmbed ?

top

You must be logged in to post comments.