Monotonicity of minimizers in optimization problems with applications to Markov control processes

Rosa M. Flores–Hernández; Raúl Montes-de-Oca

Kybernetika (2007)

  • Volume: 43, Issue: 3, page 347-368
  • ISSN: 0023-5954

Abstract

top
Firstly, in this paper there is considered a certain class of possibly unbounded optimization problems on Euclidean spaces, for which conditions that permit to obtain monotone minimizers are given. Secondly, the theory developed in the first part of the paper is applied to Markov control processes (MCPs) on real spaces with possibly unbounded cost function, and with possibly noncompact control sets, considering both the discounted and the average cost as optimality criterion. In the context described, conditions to obtain monotone optimal policies are provided. For the conditions of MCPs presented in the article, several controlled models including, in particular, two inventory/production systems and the linear regulator problem are supplied.

How to cite

top

Flores–Hernández, Rosa M., and Montes-de-Oca, Raúl. "Monotonicity of minimizers in optimization problems with applications to Markov control processes." Kybernetika 43.3 (2007): 347-368. <http://eudml.org/doc/33863>.

@article{Flores2007,
abstract = {Firstly, in this paper there is considered a certain class of possibly unbounded optimization problems on Euclidean spaces, for which conditions that permit to obtain monotone minimizers are given. Secondly, the theory developed in the first part of the paper is applied to Markov control processes (MCPs) on real spaces with possibly unbounded cost function, and with possibly noncompact control sets, considering both the discounted and the average cost as optimality criterion. In the context described, conditions to obtain monotone optimal policies are provided. For the conditions of MCPs presented in the article, several controlled models including, in particular, two inventory/production systems and the linear regulator problem are supplied.},
author = {Flores–Hernández, Rosa M., Montes-de-Oca, Raúl},
journal = {Kybernetika},
keywords = {monotone minimizer in an optimization problem; Markov control process; total discounted cost; average cost; monotone optimal policy; monotone minimizer in an optimization problem; Markov control process; total discounted cost; average cost; monotone optimal policy},
language = {eng},
number = {3},
pages = {347-368},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Monotonicity of minimizers in optimization problems with applications to Markov control processes},
url = {http://eudml.org/doc/33863},
volume = {43},
year = {2007},
}

TY - JOUR
AU - Flores–Hernández, Rosa M.
AU - Montes-de-Oca, Raúl
TI - Monotonicity of minimizers in optimization problems with applications to Markov control processes
JO - Kybernetika
PY - 2007
PB - Institute of Information Theory and Automation AS CR
VL - 43
IS - 3
SP - 347
EP - 368
AB - Firstly, in this paper there is considered a certain class of possibly unbounded optimization problems on Euclidean spaces, for which conditions that permit to obtain monotone minimizers are given. Secondly, the theory developed in the first part of the paper is applied to Markov control processes (MCPs) on real spaces with possibly unbounded cost function, and with possibly noncompact control sets, considering both the discounted and the average cost as optimality criterion. In the context described, conditions to obtain monotone optimal policies are provided. For the conditions of MCPs presented in the article, several controlled models including, in particular, two inventory/production systems and the linear regulator problem are supplied.
LA - eng
KW - monotone minimizer in an optimization problem; Markov control process; total discounted cost; average cost; monotone optimal policy; monotone minimizer in an optimization problem; Markov control process; total discounted cost; average cost; monotone optimal policy
UR - http://eudml.org/doc/33863
ER -

References

top
  1. Ash R. B., Real Variables with Basic Metric Space Topology, IEEE Press, New York 1993 Zbl0920.26002MR1193687
  2. Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F., Conditions for the uniqueness of optimal policies of discounted Markov decision processes, Math. Methods Oper. Res. 60 (2004), 415–436 Zbl1104.90053MR2106092
  3. Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F., Pointwise approximations of discounted Markov decision processes to optimal policies, Internat. J. Pure Appl. Math. 28 (2006), 265–281 Zbl1131.90068MR2228009
  4. Fu M. C., Marcus S. I., Wang, I-J, Monotone optimal policies for a transient queueing staffing problem, Oper. Res. 48 (2000), 327–331 
  5. Gallish E., On monotone optimal policies in a queueing model of M/G/1 type with controllable service time distribution, Adv. in Appl. Probab. 11 (1979), 870–887 (1979) MR0544200
  6. Hernández-Lerma O., Lasserre J. B., Discrete-Time Markov Control Processes, Springer-Verlag, New York 1996 Zbl0928.93002MR1363487
  7. Heyman D. P., Sobel M. J., Stochastic Models in Operations Research, Vol, II. Stochastic Optimization. McGraw-Hill, New York 1984 Zbl1072.90001
  8. Hinderer K., Stieglitz M., Increasing and Lipschitz continuous minimizers in one-dimensional linear-convex systems without constraints: The continuous and the discrete case, Math. Methods Oper. Res. 44 (1996), 189–204 (1996) Zbl0860.90126MR1409065
  9. Kalin D., A note on ‘monotone optimal policies for Markov decision processes’, Math. Programming 15 (1978), 220–222 (1978) Zbl0387.90106MR0509965
  10. Mendelssohn R., Sobel M., Capital accumulation and the optimization of renewable resource models, J. Econom. Theory 23 (1980), 243–260 (1980) Zbl0472.90015
  11. Pittenger A. O., Monotonicity in a Markov decision process, Math. Oper. Res. 13 (1988), 65–73 (1988) Zbl0646.90088MR0931486
  12. Porteus E. L., Foundations of Stochastic Inventory Theory, Stanford University Press, Stanford, Calif. 2002 
  13. Puterman M. L., Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York 1994 Zbl1184.90170MR1270015
  14. Rieder U., Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115–131 (1978) Zbl0385.28005MR0493590
  15. Ross S. M., Introduction to Stochastic Dynamic Programming, Academic Press, San Diego 1983 Zbl0567.90065MR0749232
  16. Serfozo R. F., Monotone optimal policies for Markov decision processes, Math. Programming Stud. 6 (1976), 202–215 (1976) Zbl0368.60080MR0459646
  17. Stidham, Sh., Weber R. R., Monotonic and insensitive optimal policies for control of queues with undiscounted costs, Oper. Res. 37 (1989), 611–625 (1989) Zbl0674.90029MR1006813
  18. Stromberg K. R., An Introduction to Classical Real Analysis, Wadsworth International Group, Belmont 1981 Zbl0454.26001MR0604364
  19. Sundaram R. K., A First Course in Optimization Theory, Cambridge University Press, Cambridge 1996 Zbl0885.90106MR1402910
  20. Topkis D. M., Minimizing a submodular function on a lattice, Oper. Res. 26 (1978), 305–321 (1978) Zbl0379.90089MR0468177
  21. Topkis D. M., Supermodularity and Complementarity, Princeton University Press, Princeton, N. J. 1988 MR1614637

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.