Discounted Markov control processes induced by deterministic systems

Hugo Cruz-Suárez; Raúl Montes-de-Oca

Kybernetika (2006)

  • Volume: 42, Issue: 6, page 647-664
  • ISSN: 0023-5954

Abstract

top
This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented.

How to cite

top

Cruz-Suárez, Hugo, and Montes-de-Oca, Raúl. "Discounted Markov control processes induced by deterministic systems." Kybernetika 42.6 (2006): 647-664. <http://eudml.org/doc/33830>.

@article{Cruz2006,
abstract = {This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented.},
author = {Cruz-Suárez, Hugo, Montes-de-Oca, Raúl},
journal = {Kybernetika},
keywords = {discounted Markov control process; deterministic control system; Euler equation; deterministic control system perturbed by a random noise; discounted Markov control process; deterministic control system; Euler equation; deterministic control system; random noise},
language = {eng},
number = {6},
pages = {647-664},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Discounted Markov control processes induced by deterministic systems},
url = {http://eudml.org/doc/33830},
volume = {42},
year = {2006},
}

TY - JOUR
AU - Cruz-Suárez, Hugo
AU - Montes-de-Oca, Raúl
TI - Discounted Markov control processes induced by deterministic systems
JO - Kybernetika
PY - 2006
PB - Institute of Information Theory and Automation AS CR
VL - 42
IS - 6
SP - 647
EP - 664
AB - This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented.
LA - eng
KW - discounted Markov control process; deterministic control system; Euler equation; deterministic control system perturbed by a random noise; discounted Markov control process; deterministic control system; Euler equation; deterministic control system; random noise
UR - http://eudml.org/doc/33830
ER -

References

top
  1. Benveniste L. M., Scheinkman J. A., 10.2307/1910417, Econometrica 47 (1979), 727–732 (1979) Zbl0435.90031MR0533081DOI10.2307/1910417
  2. Bertsekas D. P., Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, New Jersey 1987 Zbl0649.93001MR0896902
  3. Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F., 10.1007/s001860400372, Math. Methods Oper. Res. 60 (2004), 415–436 Zbl1104.90053MR2106092DOI10.1007/s001860400372
  4. Fuente A. De la, Mathematical Methods and Models for Economists, Cambridge University Press, New York 2000 Zbl0943.91001MR1735968
  5. Duffie D., Security Markets, Academic Press, Boston 1988 Zbl0861.90019MR0955269
  6. Durán J., 10.1007/s001990050016, J. Econom. Theory 15 (2000), 339–352 Zbl1101.91339MR1789235DOI10.1007/s001990050016
  7. Heer B., Maußner A., Dynamic General Equilibrium Modelling: Computational Method and Application, Springer-Verlag, Berlin 2005 MR2378171
  8. Hernández-Lerma O., Adaptive Markov Control Processes, Springer-Verlag, New York 1989 MR0995463
  9. Hernández-Lerma O., Lasserre J. B., Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, New York 1996 Zbl0840.93001MR1363487
  10. Van C. Le, Morhaim L., 10.1006/jeth.2001.2880, J. Econom. Theory 105 (2002), 158–187 Zbl1013.91079MR1912663DOI10.1006/jeth.2001.2880
  11. Levhari D., Srinivasan T. N., 10.2307/2296834, Rev. Econom. Stud. 36 (1969), 153–164 (1969) DOI10.2307/2296834
  12. Mirman L. J., Dynamic models of fishing: a heuristic approach, In: Control Theory in Mathematical Economics (Pan-Tai Liu and J. G. Sutinen, eds.), Marcel Dekker, New York 1979, pp. 39–73 (1979) Zbl0432.90024
  13. Rincón-Zapatero J. L., Rodríguez-Palmero C., 10.1111/1468-0262.00457, Econometrica 71 (2003), 1519–1555 Zbl1160.49304MR2000255DOI10.1111/1468-0262.00457
  14. Santos M. S., Numerical solution of dynamic economic models, In: Handbook of Macroeconomic, Volume I (J. B. Taylor and M. Woodford, eds.), North Holland, Amsterdam 1999, pp. 311–386 (1999) 
  15. Stokey N. L., Lucas R. E., Recursive Methods in Economic Dynamics, Harvard University Press, Cambridge, Mass. 1989 Zbl0774.90018MR1105087

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.