# Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times

Oscar Vega-Amaya; Fernando Luque-Vásquez

Applicationes Mathematicae (2000)

- Volume: 27, Issue: 3, page 343-367
- ISSN: 1233-7234

## Access Full Article

top## Abstract

top## How to cite

topVega-Amaya, Oscar, and Luque-Vásquez, Fernando. "Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times." Applicationes Mathematicae 27.3 (2000): 343-367. <http://eudml.org/doc/219278>.

@article{Vega2000,

abstract = {We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.},

author = {Vega-Amaya, Oscar, Luque-Vásquez, Fernando},

journal = {Applicationes Mathematicae},

keywords = {sample-path average costs; semi-Markov control processes},

language = {eng},

number = {3},

pages = {343-367},

title = {Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times},

url = {http://eudml.org/doc/219278},

volume = {27},

year = {2000},

}

TY - JOUR

AU - Vega-Amaya, Oscar

AU - Luque-Vásquez, Fernando

TI - Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times

JO - Applicationes Mathematicae

PY - 2000

VL - 27

IS - 3

SP - 343

EP - 367

AB - We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.

LA - eng

KW - sample-path average costs; semi-Markov control processes

UR - http://eudml.org/doc/219278

ER -

## References

top- [1] R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.
- [2] S. Bhatnagar and V. S. Borkar, A convex analytic framework for ergodic control of semi-Markov processes, Math. Oper. Res. 20 (1995), 923-936. Zbl1035.93511
- [3] R. N. Bhattacharya and M. Majumdar, Controlled semi-Markov models under long-run average rewards, J. Statist. Plann. Inference 22 (1989), 223-242. Zbl0683.49003
- [4] B. S. Borkar, Topics in Controlled Markov Chains, Pitman Res. Notes Math. Ser. 240, Longman Sci. Tech., 1991. Zbl0725.93082
- [5] R. Cavazos-Cadena and E. Fernández-Gaucherand, Denumerable controlled Markov chains with average reward criterion: Sample path optimality, Z. Oper. Res. Math. Methods Oper. Res. 41 (1995), 89-108. Zbl0835.90116
- [6] A. Federgruen, A. Hordijk and H. C. Tijms, Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion, Stochastic Process. Appl. 9 (1979), 223-235. Zbl0422.90084
- [7] A. Federgruen, P. J. Schweitzer and H. C. Tijms, Denumerable undiscounted semi-Markov decision processes with unbounded rewards, Math. Oper. Res. 8 (1983), 298-213. Zbl0513.90085
- [8] A. Federgruen and H. C. Tijms, The optimality equation in average cost denumerable state semi-Markov decision problems. Recurrence conditions and algorithms, J. Appl. Probab. 15 (1978), 356-373. Zbl0386.90060
- [9] E. A. Feinberg, Constrained semi-Markov decision processes with average rewards, Z. Oper. Res. (Math. Methods Oper. Res.) 39 (1994), 257-288. Zbl0824.90136
- [10] P. W. Glynn and S. P. Meyn, A Liapunov bound for solutions of Poisson's equation, Ann. Probab. 24 (1996), 916-931. Zbl0863.60063
- [11] E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23 (1995), 199-218. Zbl0829.93067
- [12] P. Hall and C. C. Heyde, Martingale Limit Theory and Its Application, Academic Press, 1980. Zbl0462.60045
- [13] U. G. Haussman, On the optimal long-run control of Markov renewal processes, J. Math. Anal. Appl. 36 (1971), 123-140.
- [14] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York, 1996. Zbl0840.93001
- [15] O. Hernández-Lerma and J. B. Lasserre, Further criteria for positive Harris recurrence of Markov chains, Proc. Amer. Math. Soc., to appear. Zbl0970.60078
- [16] O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality, Appl. Math. (Warsaw) 25 (1998), 153-178. Zbl0906.93062
- [17] O. Hernández-Lerma, O. Vega-Amaya and G. Carrasco, Sample-path optimality and variance-minimization of average cost Markov control processes, SIAM J. Control Optim., to appear. Zbl0951.93074
- [18] M. Kurano, Semi-Markov decision processes and their applications in replacement models, J. Oper. Res. Soc. Japan 28 (1985), 18-29. Zbl0564.90090
- [19] M. Kurano, Average optimal adaptive policies in semi-Markov decision processes including an unknown parameter, ibid., 252-266. Zbl0579.90098
- [20] J. B. Lasserre, Sample-path average optimality for Markov control processes, IEEE Trans. Automat. Control, to appear. Zbl0956.93066
- [21] S. A. Lippman, Semi-Markov decision processes with unbounded rewards, Management Sci. 19 (1973), 717-731. Zbl0259.60044
- [22] S. A. Lippman, On dynamic programming with unbounded rewards, ibid. 21 (1975), 1225-1233. Zbl0309.90017
- [23] F. Luque-Vásquez and O. Hernández-Lerma, Semi-Markov control models with average costs, Appl. Math. (Warsaw) 26 (1999), 315-331. Zbl1050.90566
- [24] S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer, London, 1993. Zbl0925.60001
- [25] M. L. Puterman, Markov Decision Processes, Wiley, New York, 1994.
- [26] S. M. Ross, Average cost semi-Markov decision processes, J. Appl. Probab. 7 (1979), 649-656. Zbl0204.51704
- [27] M. Schäl, On the second optimality equation for semi-Markov decision models, Math. Oper. Res. 17 (1992), 470-486. Zbl0773.90091
- [28] P. J. Schweitzer, Iterative solutions of the functional equations of undiscounted Markov renewal programming, J. Math. Anal. Appl. 34 (1971), 495-501. Zbl0218.90070
- [29] L. I. Sennott, Average cost semi-Markov decision processes and the control of queueing systems, Probab. Engrg. Inform. Sci. 3 (1989), 247-272. Zbl1134.60408
- [30] O. Vega-Amaya, Sample path average optimality of Markov control processes with strictly unbounded cost, Appl. Math. (Warsaw) 26 (1999), 363-381. Zbl1050.93523
- [31] O. Vega-Amaya, Markov control processes in Borel spaces: undiscounted cost criteria, doctoral thesis, UAM-Iztapalapa, México, 1998 (in Spanish).

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.