Stability estimating in optimal stopping problem

Elena Zaitseva

Kybernetika (2008)

  • Volume: 44, Issue: 3, page 400-415
  • ISSN: 0023-5954

Abstract

top
We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space X . It is supposed that an unknown transition probability p ( · | x ) , x ∈ X , is approximated by the transition probability p ˜ ( · | x ) , x ∈ X , and the stopping rule τ ˜ * , optimal for p ˜ , is applied to the process governed by p . We found an upper bound for the difference between the total expected cost, resulting when applying τ ˜ * , and the minimal total expected cost. The bound given is a constant times sup x ∈ X ∥ p ( · | x ) - p ˜ ( · | x ) ∥ , where ∥ · ∥ is the total variation norm.

How to cite

top

Zaitseva, Elena. "Stability estimating in optimal stopping problem." Kybernetika 44.3 (2008): 400-415. <http://eudml.org/doc/33936>.

@article{Zaitseva2008,
abstract = {We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space $X$. It is supposed that an unknown transition probability $p(\cdot |x)$, $x\in X$, is approximated by the transition probability $\widetilde\{p\}(\cdot |x)$, $x\in X$, and the stopping rule $\widetilde\{\tau \}_*$, optimal for $\widetilde\{p\}$, is applied to the process governed by $p$. We found an upper bound for the difference between the total expected cost, resulting when applying $\widetilde\{\tau \}_*$, and the minimal total expected cost. The bound given is a constant times $\displaystyle \sup \nolimits _\{x\in X\}\Vert p(\cdot |x)-\widetilde\{p\}(\cdot |x)\Vert $, where $\Vert \cdot \Vert $ is the total variation norm.},
author = {Zaitseva, Elena},
journal = {Kybernetika},
keywords = {discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling; discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling},
language = {eng},
number = {3},
pages = {400-415},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Stability estimating in optimal stopping problem},
url = {http://eudml.org/doc/33936},
volume = {44},
year = {2008},
}

TY - JOUR
AU - Zaitseva, Elena
TI - Stability estimating in optimal stopping problem
JO - Kybernetika
PY - 2008
PB - Institute of Information Theory and Automation AS CR
VL - 44
IS - 3
SP - 400
EP - 415
AB - We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space $X$. It is supposed that an unknown transition probability $p(\cdot |x)$, $x\in X$, is approximated by the transition probability $\widetilde{p}(\cdot |x)$, $x\in X$, and the stopping rule $\widetilde{\tau }_*$, optimal for $\widetilde{p}$, is applied to the process governed by $p$. We found an upper bound for the difference between the total expected cost, resulting when applying $\widetilde{\tau }_*$, and the minimal total expected cost. The bound given is a constant times $\displaystyle \sup \nolimits _{x\in X}\Vert p(\cdot |x)-\widetilde{p}(\cdot |x)\Vert $, where $\Vert \cdot \Vert $ is the total variation norm.
LA - eng
KW - discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling; discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling
UR - http://eudml.org/doc/33936
ER -

References

top
  1. Allart P., Optimal stopping rules for correlated random walks with a discount, J. Appl. Prob. 41 (2004), 483–496 MR2052586
  2. Bertsekas D. P., Dynamic Programming: Deterministic and Stochastic Models, Prentice Hall, Englewood Cliffs, N. J. 1987 Zbl0649.93001MR0896902
  3. Bertsekas D. P., Shreve S. E., Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York 1979 Zbl0633.93001MR0511544
  4. Dijk N. M. Van, Perturbation theory for unbounded Markov reward process with applications to queueing systems, Adv. in Appl. Probab. 20 (1988), 99–111 (1988) MR0932536
  5. Dijk N. M. Van, Sladký K., Error bounds for nonnegative dynamic models, J. Optim. Theory Appl. 101 (1999), 449–474 (1999) MR1684679
  6. Dynkin E. B., Yushkevich A. A., Controlled Markov Process, Springer-Verlag, New York 1979 MR0554083
  7. Favero G., Runggaldier W. J., A robustness results for stochastic control, Systems Control Lett. 46 (2002), 91–97 MR2010062
  8. Gordienko E. I., An estimate of the stability of optimal control of certain stochastic and deterministic systems, J. Soviet Math. 59 (1992), 891–899. (Translated from the Russian publication of 1989) (1992) MR1163393
  9. Gordienko E. I., Salem F. S., Robustness inequality for Markov control process with unbounded costs, Systems Control Lett. 33 (1998), 125–130 (1998) MR1607814
  10. Gordienko E. I., Yushkevich A. A., Stability estimates in the problem of average optimal switching of a Markov chain, Math. Methods Oper. Res. 57 (2003), 345–365 Zbl1116.90401MR1990916
  11. Gordienko E. I., Lemus-Rodríguez E., Montes-de-Oca R., Discounted cost optimality problem: stability with respect to weak metrics, In press in: Math. Methhods Oper. Res. (2008) Zbl1166.60041MR2429561
  12. Hernández-Lerma O., Lassere J. B., Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, N.Y. 1996 
  13. Jensen U., An optimal stopping problem in risk theory, Scand. Actuarial J.2 (1997), 149–159 (1997) Zbl0888.62104MR1492423
  14. Meyn S. P., Tweedie R. L., Markov Chains and Stochastic Stability, Springer-Verlag, London 1993 Zbl1165.60001MR1287609
  15. Montes-de-Oca R., Salem-Silva F., Estimates for perturbations of an average Markov decision process with a minimal state and upper bounded by stochastically ordered Markov chains, Kybernetika 41 (2005), 757–772 MR2193864
  16. Montes-de-Oca R., Sakhanenko, A., Salem-Silva F., Estimate for perturbations of general discounted Markov control chains, Appl. Math. 30 (2003), 287–304 MR2029538
  17. Muciek B. K., Optimal stopping of a risk process: model with interest rates, J. Appl. Prob. 39 (2002), 261–270 Zbl1011.62111MR1908943
  18. Müller A., How does the value function of a Markov decision process depend on the transition probabilities? Math, Oper. Res. 22 (1997), 872–885 (1997) MR1484687
  19. Schäl M., Conditions for optimality in dynamic programming and for the limit of n -stage optimal policies to be optimal, Z. Wahrsch. verw. Gebiete 32 (1975), 179–196 (1975) Zbl0316.90080MR0378841
  20. Shiryaev A. N., Optimal Stopping Rules, Springer-Verlag, New York 1978 Zbl1138.60008MR2374974
  21. Shiryaev A. N., Essential of Stochastic Finance, Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N.J. 1999 MR1695318

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.