Risk probability optimization problem for finite horizon continuous time Markov decision processes with loss rate

Haifeng Huo; Xian Wen

Kybernetika (2021)

  • Issue: 2, page 272-294
  • ISSN: 0023-5954

Abstract

top
This paper presents a study the risk probability optimality for finite horizon continuous-time Markov decision process with loss rate and unbounded transition rates. Under drift condition, which is slightly weaker than the regular condition, as detailed in existing literature on the risk probability optimality Semi-Markov decision processes, we prove that the value function is the unique solution of the corresponding optimality equation, and demonstrate the existence of a risk probability optimization policy using an iteration technique. Furthermore, we provide verification of the imposed condition with two examples of controlled birth-and-death system and risk control, and further demonstrate that a value iteration algorithm can be used to calculate the value function and develop an optimal policy.

How to cite

top

Huo, Haifeng, and Wen, Xian. "Risk probability optimization problem for finite horizon continuous time Markov decision processes with loss rate." Kybernetika (2021): 272-294. <http://eudml.org/doc/297714>.

@article{Huo2021,
abstract = {This paper presents a study the risk probability optimality for finite horizon continuous-time Markov decision process with loss rate and unbounded transition rates. Under drift condition, which is slightly weaker than the regular condition, as detailed in existing literature on the risk probability optimality Semi-Markov decision processes, we prove that the value function is the unique solution of the corresponding optimality equation, and demonstrate the existence of a risk probability optimization policy using an iteration technique. Furthermore, we provide verification of the imposed condition with two examples of controlled birth-and-death system and risk control, and further demonstrate that a value iteration algorithm can be used to calculate the value function and develop an optimal policy.},
author = {Huo, Haifeng, Wen, Xian},
journal = {Kybernetika},
keywords = {continuous-time Markov decision processes; loss rate; risk probability criterion; finite horizon; optimal policy; unbounded transition rate},
language = {eng},
number = {2},
pages = {272-294},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Risk probability optimization problem for finite horizon continuous time Markov decision processes with loss rate},
url = {http://eudml.org/doc/297714},
year = {2021},
}

TY - JOUR
AU - Huo, Haifeng
AU - Wen, Xian
TI - Risk probability optimization problem for finite horizon continuous time Markov decision processes with loss rate
JO - Kybernetika
PY - 2021
PB - Institute of Information Theory and Automation AS CR
IS - 2
SP - 272
EP - 294
AB - This paper presents a study the risk probability optimality for finite horizon continuous-time Markov decision process with loss rate and unbounded transition rates. Under drift condition, which is slightly weaker than the regular condition, as detailed in existing literature on the risk probability optimality Semi-Markov decision processes, we prove that the value function is the unique solution of the corresponding optimality equation, and demonstrate the existence of a risk probability optimization policy using an iteration technique. Furthermore, we provide verification of the imposed condition with two examples of controlled birth-and-death system and risk control, and further demonstrate that a value iteration algorithm can be used to calculate the value function and develop an optimal policy.
LA - eng
KW - continuous-time Markov decision processes; loss rate; risk probability criterion; finite horizon; optimal policy; unbounded transition rate
UR - http://eudml.org/doc/297714
ER -

References

top
  1. Boda, K., Filar, J. A., Lin, Y. L., , IEEE Trans. Automat. Control 49 (2004), 409-419. MR2062253DOI
  2. Bouakiz, M., Kebir, Y., , J. Optim. Theory Appl. 86 (1995), 1-15. MR1341504DOI
  3. Bertsekas, D., Shreve, S., Stochastic Optimal Control: The Discrete-Time Case., Academic Press Inc, New York 1978 MR0511544
  4. Bauerle, N., Rieder, U., Markov Decision Processes with Applications to Finance., Springer, Heidelberg 2011 MR2808878
  5. Feinberg, E., , Math. Operat. Res. 29 (2004), 492-524. MR2082616DOI
  6. Guo, X. P., Hernández-Lerma, O., Continuous-Time Markov Decision Process: Theorey and Applications., Springer-Verlag, Berlin 2009. MR2554588
  7. Guo, X. P., Piunovskiy, A., , Math. Oper. Res. 36 (2011), 105-132. MR2799395DOI
  8. Guo, X. P., Huang, X. X., Huang, Y. H., , Adv. Appl. Prob. 47 (2015), 1064-1087. MR3433296DOI
  9. Hernández-Lerma, O., Lasserre, J. B., Discrete-Time Markov Control Process: Basic Optimality Criteria., Springer-Verlag, New York 1996. MR1363487
  10. Huang, Y. H., Guo, X. P., , J. Math. Anal. Appl. 359 (2009), 404-420. MR2542184DOI
  11. Huang, Y. H., Guo, X. P., , Acta. Math. Appl. Sinica 27 (2011), 177-190. MR2784052DOI
  12. Huang, Y. H., Guo, X. P., Li, Z. F., , J. Math. Anal. Appl. 402 (2013), 378-391. MR3023265DOI
  13. Huang, X. X., Zou, X. L., Guo, X. P., , Sci. China Math. 58 (2015), 1923-1938. MR3383991DOI
  14. Huo, H. F., Zou, X. L., Guo, X. P., , Discrete Event Dynamic system: Theory Appl. 27 (2017), 675-699. MR3712415DOI
  15. Huo, H. F., Wen, X., , Kybernetika 55 (2019), 114-133. MR3935417DOI
  16. Huo, H. F., Guo, X.P., , IEEE trans. Automat. Control 65 (2020), 3199-3206. MR4120586DOI
  17. Jacod, J., , Z. Wahrscheinlichkeitstheorie und verwandte Gebiete 31 (1975), 235-253. MR0380978DOI
  18. Janssen, J., Manca, R., Semi-Markov Risk Models For Finance, Insurance, and Reliability., Springer-Verlag, New York 2006. MR2301626
  19. Liu, Q. L., Zou, X. L., , J. Dynamics Games 5 (2018), 143-163. MR3810203DOI
  20. Piunovskiy, A., Zhang, Y., , SIAM J. Control Optim. 49 (2011), 2032-2061. MR2837510DOI
  21. Ohtsubo, Y., Toyonaga, K., , J. Math. Anal. Appl. 271 (2002), 66-81. MR1923747DOI
  22. Ohtsubo, Y., , J. Oper. Res. Soc. Japan 46 (2003), 342-352. MR2011960DOI
  23. Ohtsubo, Y., Toyonaga, K., , Math. Methods Oper. Res. 60 (2004), 239-250. MR2099534DOI
  24. Puterman, M. L., Markov Decision Processes: Discrete Stochastic Dynamic Programming., John Wiley, New York 1994. Zbl1184.90170MR1270015
  25. Sakaguchi, M., Ohtsubo, Y., , Appl. Math. Comput. 216 (2010), 2947-2958. MR2653110DOI
  26. Sobel, M. J., 10.2307/3213832, J. Appl. Probab. 19 (1982), 744-802. Zbl0503.90091MR0675143DOI10.2307/3213832
  27. Wei, Q. D., Guo, X. P., , Optimization 64 (2015), 1593-1623. MR3340650DOI
  28. White, D. J., , J. Math. Anal. Appl. Optim. 173 (1993), 634-646. MR1209345DOI
  29. Wu, C. B., Lin, Y. L., , J. Math. Anal. Appl. 231 (1999), 47-67. MR1676741DOI
  30. Wu, R., Fang, K., , Acta Math. Applic. Sinica 15 (1999), 352-360. MR1735505DOI
  31. Yu, S. X., Lin, Y. L., Yan, P. F., , J. Math. Anal. Appl. 225 (1998), 193-223. MR1639236DOI
  32. Xia, L., , Automatica 73 (2016), 269-278. MR3552085DOI

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.