Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times

Oscar Vega-Amaya; Fernando Luque-Vásquez

Applicationes Mathematicae (2000)

  • Volume: 27, Issue: 3, page 343-367
  • ISSN: 1233-7234

Abstract

top
We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.

How to cite

top

Vega-Amaya, Oscar, and Luque-Vásquez, Fernando. "Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times." Applicationes Mathematicae 27.3 (2000): 343-367. <http://eudml.org/doc/219278>.

@article{Vega2000,
abstract = {We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.},
author = {Vega-Amaya, Oscar, Luque-Vásquez, Fernando},
journal = {Applicationes Mathematicae},
keywords = {sample-path average costs; semi-Markov control processes},
language = {eng},
number = {3},
pages = {343-367},
title = {Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times},
url = {http://eudml.org/doc/219278},
volume = {27},
year = {2000},
}

TY - JOUR
AU - Vega-Amaya, Oscar
AU - Luque-Vásquez, Fernando
TI - Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times
JO - Applicationes Mathematicae
PY - 2000
VL - 27
IS - 3
SP - 343
EP - 367
AB - We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.
LA - eng
KW - sample-path average costs; semi-Markov control processes
UR - http://eudml.org/doc/219278
ER -

References

top
  1. [1] R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972. 
  2. [2] S. Bhatnagar and V. S. Borkar, A convex analytic framework for ergodic control of semi-Markov processes, Math. Oper. Res. 20 (1995), 923-936. Zbl1035.93511
  3. [3] R. N. Bhattacharya and M. Majumdar, Controlled semi-Markov models under long-run average rewards, J. Statist. Plann. Inference 22 (1989), 223-242. Zbl0683.49003
  4. [4] B. S. Borkar, Topics in Controlled Markov Chains, Pitman Res. Notes Math. Ser. 240, Longman Sci. Tech., 1991. Zbl0725.93082
  5. [5] R. Cavazos-Cadena and E. Fernández-Gaucherand, Denumerable controlled Markov chains with average reward criterion: Sample path optimality, Z. Oper. Res. Math. Methods Oper. Res. 41 (1995), 89-108. Zbl0835.90116
  6. [6] A. Federgruen, A. Hordijk and H. C. Tijms, Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion, Stochastic Process. Appl. 9 (1979), 223-235. Zbl0422.90084
  7. [7] A. Federgruen, P. J. Schweitzer and H. C. Tijms, Denumerable undiscounted semi-Markov decision processes with unbounded rewards, Math. Oper. Res. 8 (1983), 298-213. Zbl0513.90085
  8. [8] A. Federgruen and H. C. Tijms, The optimality equation in average cost denumerable state semi-Markov decision problems. Recurrence conditions and algorithms, J. Appl. Probab. 15 (1978), 356-373. Zbl0386.90060
  9. [9] E. A. Feinberg, Constrained semi-Markov decision processes with average rewards, Z. Oper. Res. (Math. Methods Oper. Res.) 39 (1994), 257-288. Zbl0824.90136
  10. [10] P. W. Glynn and S. P. Meyn, A Liapunov bound for solutions of Poisson's equation, Ann. Probab. 24 (1996), 916-931. Zbl0863.60063
  11. [11] E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23 (1995), 199-218. Zbl0829.93067
  12. [12] P. Hall and C. C. Heyde, Martingale Limit Theory and Its Application, Academic Press, 1980. Zbl0462.60045
  13. [13] U. G. Haussman, On the optimal long-run control of Markov renewal processes, J. Math. Anal. Appl. 36 (1971), 123-140. 
  14. [14] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York, 1996. Zbl0840.93001
  15. [15] O. Hernández-Lerma and J. B. Lasserre, Further criteria for positive Harris recurrence of Markov chains, Proc. Amer. Math. Soc., to appear. Zbl0970.60078
  16. [16] O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality, Appl. Math. (Warsaw) 25 (1998), 153-178. Zbl0906.93062
  17. [17] O. Hernández-Lerma, O. Vega-Amaya and G. Carrasco, Sample-path optimality and variance-minimization of average cost Markov control processes, SIAM J. Control Optim., to appear. Zbl0951.93074
  18. [18] M. Kurano, Semi-Markov decision processes and their applications in replacement models, J. Oper. Res. Soc. Japan 28 (1985), 18-29. Zbl0564.90090
  19. [19] M. Kurano, Average optimal adaptive policies in semi-Markov decision processes including an unknown parameter, ibid., 252-266. Zbl0579.90098
  20. [20] J. B. Lasserre, Sample-path average optimality for Markov control processes, IEEE Trans. Automat. Control, to appear. Zbl0956.93066
  21. [21] S. A. Lippman, Semi-Markov decision processes with unbounded rewards, Management Sci. 19 (1973), 717-731. Zbl0259.60044
  22. [22] S. A. Lippman, On dynamic programming with unbounded rewards, ibid. 21 (1975), 1225-1233. Zbl0309.90017
  23. [23] F. Luque-Vásquez and O. Hernández-Lerma, Semi-Markov control models with average costs, Appl. Math. (Warsaw) 26 (1999), 315-331. Zbl1050.90566
  24. [24] S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer, London, 1993. Zbl0925.60001
  25. [25] M. L. Puterman, Markov Decision Processes, Wiley, New York, 1994. 
  26. [26] S. M. Ross, Average cost semi-Markov decision processes, J. Appl. Probab. 7 (1979), 649-656. Zbl0204.51704
  27. [27] M. Schäl, On the second optimality equation for semi-Markov decision models, Math. Oper. Res. 17 (1992), 470-486. Zbl0773.90091
  28. [28] P. J. Schweitzer, Iterative solutions of the functional equations of undiscounted Markov renewal programming, J. Math. Anal. Appl. 34 (1971), 495-501. Zbl0218.90070
  29. [29] L. I. Sennott, Average cost semi-Markov decision processes and the control of queueing systems, Probab. Engrg. Inform. Sci. 3 (1989), 247-272. Zbl1134.60408
  30. [30] O. Vega-Amaya, Sample path average optimality of Markov control processes with strictly unbounded cost, Appl. Math. (Warsaw) 26 (1999), 363-381. Zbl1050.93523
  31. [31] O. Vega-Amaya, Markov control processes in Borel spaces: undiscounted cost criteria, doctoral thesis, UAM-Iztapalapa, México, 1998 (in Spanish). 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.