About stability of risk-seeking optimal stopping

Raúl Montes-de-Oca; Elena Zaitseva

Kybernetika (2014)

  • Volume: 50, Issue: 3, page 378-392
  • ISSN: 0023-5954

Abstract

top
We offer the quantitative estimation of stability of risk-sensitive cost optimization in the problem of optimal stopping of Markov chain on a Borel space X . It is supposed that the transition probability p ( · | x ) , x ∈ X is approximated by the transition probability p ˜ ( · | x ) , x ∈ X , and that the stopping rule f ˜ * , which is optimal for the process with the transition probability p ˜ is applied to the process with the transition probability p . We give an upper bound (expressed in term of the total variation distance: sup x ∈ X ∥ p ( · | x ) - p ˜ ( · | x ) ∥ ) for an additional cost paid for using the rule f ˜ * instead of the (unknown) stopping rule f * optimal for p .

How to cite

top

Montes-de-Oca, Raúl, and Zaitseva, Elena. "About stability of risk-seeking optimal stopping." Kybernetika 50.3 (2014): 378-392. <http://eudml.org/doc/261895>.

@article{Montes2014,
abstract = {We offer the quantitative estimation of stability of risk-sensitive cost optimization in the problem of optimal stopping of Markov chain on a Borel space $X$. It is supposed that the transition probability $p(\cdot |x)$, $x\in X$ is approximated by the transition probability $\widetilde\{p\}(\cdot |x)$, $x\in X$, and that the stopping rule $\widetilde\{f\}_*$ , which is optimal for the process with the transition probability $\widetilde\{p\}$ is applied to the process with the transition probability $p$. We give an upper bound (expressed in term of the total variation distance: $\sup _\{x\in X\}\Vert p(\cdot |x)-\widetilde\{p\}(\cdot |x)\Vert )$ for an additional cost paid for using the rule $\widetilde\{f\}_*$ instead of the (unknown) stopping rule $f_*$ optimal for $p$.},
author = {Montes-de-Oca, Raúl, Zaitseva, Elena},
journal = {Kybernetika},
keywords = {discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric; discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric},
language = {eng},
number = {3},
pages = {378-392},
publisher = {Institute of Information Theory and Automation AS CR},
title = {About stability of risk-seeking optimal stopping},
url = {http://eudml.org/doc/261895},
volume = {50},
year = {2014},
}

TY - JOUR
AU - Montes-de-Oca, Raúl
AU - Zaitseva, Elena
TI - About stability of risk-seeking optimal stopping
JO - Kybernetika
PY - 2014
PB - Institute of Information Theory and Automation AS CR
VL - 50
IS - 3
SP - 378
EP - 392
AB - We offer the quantitative estimation of stability of risk-sensitive cost optimization in the problem of optimal stopping of Markov chain on a Borel space $X$. It is supposed that the transition probability $p(\cdot |x)$, $x\in X$ is approximated by the transition probability $\widetilde{p}(\cdot |x)$, $x\in X$, and that the stopping rule $\widetilde{f}_*$ , which is optimal for the process with the transition probability $\widetilde{p}$ is applied to the process with the transition probability $p$. We give an upper bound (expressed in term of the total variation distance: $\sup _{x\in X}\Vert p(\cdot |x)-\widetilde{p}(\cdot |x)\Vert )$ for an additional cost paid for using the rule $\widetilde{f}_*$ instead of the (unknown) stopping rule $f_*$ optimal for $p$.
LA - eng
KW - discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric; discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric
UR - http://eudml.org/doc/261895
ER -

References

top
  1. Avila-Godoy, G., Fernández-Gaucherand, E., Controlled Markov chains with exponential risk-sensitive criteria: modularity, structured policies and applications., In: Decision and Control 1998. Proc. 37th IEEE Conference. Vol. 1, IEEE, pp. 778-783. 
  2. Bäuerle, N., Rieder, U., Markov Decision Processes with Applications to Finance., Springer-Verlag, Berlin 2011. Zbl1236.90004MR2808878
  3. Borkar, V. S., Meyn, S. P., 10.1287/moor.27.1.192.334, Math. Oper. Res. 27 (2002), 192-209. Zbl1082.90577MR1886226DOI10.1287/moor.27.1.192.334
  4. Cavazos-Cadena, R., 10.1007/s00186-009-0285-6, Math. Methods Oper. Res. 71 (2010), 47-84. Zbl1189.93144MR2595908DOI10.1007/s00186-009-0285-6
  5. Cavazos-Cadena, R., Fernández-Gaucherand, E., Controlled Markov chains with risk-sensitive criteria: Average costs, optimality equations, and optimal solutions., Math. Methods Oper. Res. 49 (1999), 299-324. MR1687362
  6. Cavazos-Cadena, R., Montes-de-Oca, R., Optimal stationary policies in risk-sensitive dynamic programs with finite state space and nonegative rewards., Appl. Math. 27 (2000), 167-185. MR1768711
  7. Dijk, N. M. Van, Sladký, K., 10.1023/A:1021749829267, J. Optim. Theory Appl. 101 (1999), 449-474. MR1684679DOI10.1023/A:1021749829267
  8. Devroye, L., Györfy, L., Nonparametric Density Estimation: The L 1 View., John Wiley, New York 1986. 
  9. Dynkin, E. B., Yushkevich, A. A., Controlled Markov Processes., Springer Verlag, New York 1979. MR0554083
  10. Gordienko, E. I., Yushkevich, A. A., Stability estimates in the problem of average optimal switching of a Markov chain., Math. Methods Oper. Res. 57 (2003), 345-365. Zbl1116.90401MR1990916
  11. Gordienko, E. I., Lemus-Rodríguez, E., Montes-de-Oca, R., 10.1007/s00186-008-0229-6, Math. Methods Oper. Res. 70 (2009), 13-33. Zbl1176.60062MR2529423DOI10.1007/s00186-008-0229-6
  12. Gordienko, E. I., Salem, F., 10.1016/S0167-6911(97)00077-7, Syst. Control Lett. 33 (1998), 125-130. MR1607814DOI10.1016/S0167-6911(97)00077-7
  13. Hernández-Lerma, O., Lasserre, J. B., Further Topics on Discrete-time Markov Control Processes., Springer-Verlag, New York 1999. Zbl0928.93002MR1697198
  14. Jaśkiewicz, A., 10.1214/105051606000000790, Ann. Appl. Probab. 17 (2007), 654-675. Zbl1128.93056MR2308338DOI10.1214/105051606000000790
  15. Kartashov, N. V., Strong Stable Markov Chains., VSP, Utrecht 1996. Zbl0874.60082MR1451375
  16. Marcus, S. I., Fernández-Gaucherand, E., Hernández-Hernández, D. E., Coraluppi, S., Fard, P., Risk sensitive Markov decision processes., Progress in System and Control Theory 22 (1997), 263-280. MR1427787
  17. Masi, G. B. Di, Stettner, L., 10.1016/S0167-6911(99)00118-8, Systems Control Lett. 40 (2000), 15-20. Zbl0977.93083MR1829070DOI10.1016/S0167-6911(99)00118-8
  18. Meyn, S. P., Tweedie, R. L., Markov Chains and Stochastic Stability., Springer-Verlag, London 1993. Zbl1165.60001MR1287609
  19. Montes-de-Oca, R., Salem-Silva, F., Estimates for perturbations of an average Markov decision processes with a minimal state and upper bounded stochastically ordered Markov chains., Kybernetika 41 (2005), 757-772. MR2193864
  20. Muciek, B. K., 10.1239/jap/1025131424, J. Appl. Probab. 39 (2002), 261-270. MR1908943DOI10.1239/jap/1025131424
  21. Shiryaev, A. N., Optimal Stopping Rules., Springer-Verlag, New York 1978. Zbl1138.60008MR2374974
  22. Shiryaev, A. N., Essential of Stochastic Finance. Facts, Models, Theory., World Scientific Publishing Co., Inc., River Edge, N. J. 1999. MR1695318
  23. Sladký, K., Bounds on discrete dynamic programming recursions I., Kybernetika 16 (1980), 526-547. Zbl0454.90085MR0607292
  24. Zaitseva, E., Stability estimating in optimal stopping problem., Kybernetika 44 (2008), 400-415. Zbl1154.60326MR2436040

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.