Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion

Juan González-Hernández; Raquiel R. López-Martínez; J. Adolfo Minjárez-Sosa

Kybernetika (2009)

  • Volume: 45, Issue: 5, page 737-754
  • ISSN: 0023-5954

Abstract

top
The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process x t and the discount process α t evolve according to the coupled difference equations x t + 1 = F ( x t , α t , a t , ξ t ) , α t + 1 = G ( α t , η t ) where the state and discount disturbance processes { ξ t } and { η t } are sequences of i.i.d. random variables with densities ρ ξ and ρ η respectively. The main objective is to introduce approximation algorithms of the optimal cost function that lead up to construction of optimal or nearly optimal policies in the cases when the densities ρ ξ and ρ η are either known or unknown. In the latter case, we combine suitable estimation methods with control procedures to construct an asymptotically discounted optimal policy.

How to cite

top

González-Hernández, Juan, López-Martínez, Raquiel R., and Minjárez-Sosa, J. Adolfo. "Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion." Kybernetika 45.5 (2009): 737-754. <http://eudml.org/doc/37698>.

@article{González2009,
abstract = {The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process $\left\lbrace x_\{t\}\right\rbrace $ and the discount process $\left\lbrace \alpha _\{t\}\right\rbrace $ evolve according to the coupled difference equations $x_\{t+1\}=F(x_\{t\},\alpha _\{t\},a_\{t\},\xi _\{t\}),$$ \alpha _\{t+1\}=G(\alpha _\{t\},\eta _\{t\})$ where the state and discount disturbance processes $\lbrace \xi _\{t\}\rbrace $ and $\lbrace \eta _\{t\}\rbrace $ are sequences of i.i.d. random variables with densities $\rho ^\{\xi \}$ and $\rho ^\{\eta \}$ respectively. The main objective is to introduce approximation algorithms of the optimal cost function that lead up to construction of optimal or nearly optimal policies in the cases when the densities $\rho ^\{\xi \}$ and $\rho ^\{\eta \}$ are either known or unknown. In the latter case, we combine suitable estimation methods with control procedures to construct an asymptotically discounted optimal policy.},
author = {González-Hernández, Juan, López-Martínez, Raquiel R., Minjárez-Sosa, J. Adolfo},
journal = {Kybernetika},
keywords = {discounted cost; random rate; stochastic systems; approximation algorithms; density estimation; density estimation; stochastic systems; discounted cost; random rate; approximation algorithms},
language = {eng},
number = {5},
pages = {737-754},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion},
url = {http://eudml.org/doc/37698},
volume = {45},
year = {2009},
}

TY - JOUR
AU - González-Hernández, Juan
AU - López-Martínez, Raquiel R.
AU - Minjárez-Sosa, J. Adolfo
TI - Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion
JO - Kybernetika
PY - 2009
PB - Institute of Information Theory and Automation AS CR
VL - 45
IS - 5
SP - 737
EP - 754
AB - The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process $\left\lbrace x_{t}\right\rbrace $ and the discount process $\left\lbrace \alpha _{t}\right\rbrace $ evolve according to the coupled difference equations $x_{t+1}=F(x_{t},\alpha _{t},a_{t},\xi _{t}),$$ \alpha _{t+1}=G(\alpha _{t},\eta _{t})$ where the state and discount disturbance processes $\lbrace \xi _{t}\rbrace $ and $\lbrace \eta _{t}\rbrace $ are sequences of i.i.d. random variables with densities $\rho ^{\xi }$ and $\rho ^{\eta }$ respectively. The main objective is to introduce approximation algorithms of the optimal cost function that lead up to construction of optimal or nearly optimal policies in the cases when the densities $\rho ^{\xi }$ and $\rho ^{\eta }$ are either known or unknown. In the latter case, we combine suitable estimation methods with control procedures to construct an asymptotically discounted optimal policy.
LA - eng
KW - discounted cost; random rate; stochastic systems; approximation algorithms; density estimation; density estimation; stochastic systems; discounted cost; random rate; approximation algorithms
UR - http://eudml.org/doc/37698
ER -

References

top
  1. The effects of different inflation risk premiums on interest rate spreads, Physica A 333 (2004), 317–324. MR2100223
  2. Nonparametric Density Estimation the L 1 View, Wiley, New York 1985. MR0780746
  3. Controlled Markov Processes, Springer–Verlag, New York 1979. MR0554083
  4. [unknown], A. Gil and A. Luis: Modelling the U. S. interest rate in terms of I(d) statistical model. Quart. Rev. Economics and Finance 44 (2004), 475–486. 
  5. On density estimation in the view of Kolmogorov’s ideas in approximation theory, Ann. Statist. 18 (1990), 999–1010. MR1062695
  6. Markov control processes with randomized discounted cost in Borel space, Math. Meth. Oper. Res. 65 (2007), 27–44. 
  7. Optimal pension funding dynamics over infinite control horizon when stochastic rates of return are stationary, Insurance Math. Econom. 36 (2005), 103–116. MR2122668
  8. Adaptive Markov Control Processes, Springer–Verlag, New York 1989. MR0995463
  9. Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer–Verlag, New York 1996. MR1363487
  10. Further Topics on Discrete-Time Markov Control Processes, Springer–Verlag, New York 1999. MR1697198
  11. Monotone approximations for convex stochastic control problems, J. Math. Syst., Estimation, and Control 4 (1994), 99–140. MR1298550
  12. When to refinance a mortgage: a dynamic programming approach, European J. Oper. Res. 166 (2005), 266–277. 
  13. Discounting the distant future: how much do uncertain rates increase valuation? J, Environmental Economic and Management 46 (2003), 52–71. 
  14. Interest-rate smooothing and optimal monetary policy: A review of recent empirical evidence, J. Econom. Business 52 (2000), 205–228. 
  15. Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gerb. 32 (1975), 179–196. MR0378841
  16. Estimation and control in discounted stochastic dynamic programming, Stochastics 20 (1987), 51–71. MR0875814
  17. Recursive Methods in Economic Dynamics, Harvard University Press, Cambridge, MA 1989. MR1105087

Citations in EuDML Documents

top
  1. Irina Bashkirtseva, Controlling the stochastic sensitivity in thermochemical systems under incomplete information
  2. Yofre H. García, Juan González-Hernández, Discrete-time Markov control processes with recursive discount rates
  3. Rocio Ilhuicatzi-Roldán, Hugo Cruz-Suárez, Selene Chávez-Rodríguez, Markov decision processes with time-varying discount factors and random horizon
  4. Beatris A. Escobedo-Trujillo, Carmen G. Higuera-Chan, Time-varying Markov decision processes with state-action-dependent discount factors and unbounded costs
  5. E. Everardo Martinez-Garcia, J. Adolfo Minjárez-Sosa, Oscar Vega-Amaya, Partially observable Markov decision processes with partially observable random discount factors

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.