An extended version of average Markov decision processes on discrete spaces under fuzzy environment

Hugo Cruz-Suárez; Raúl Montes-de-Oca; R. Israel Ortega-Gutiérrez

Kybernetika (2023)

  • Volume: 59, Issue: 1, page 160-178
  • ISSN: 0023-5954

Abstract

top
The article presents an extension of the theory of standard Markov decision processes on discrete spaces and with the average cost as the objective function which permits to take into account a fuzzy average cost of a trapezoidal type. In this context, the fuzzy optimal control problem is considered with respect to two cases: the max-order of the fuzzy numbers and the average ranking order of the trapezoidal fuzzy numbers. Each of these cases extends the standard optimal control problem, and for each of them the optimal solution is related to a suitable standard optimal control problem, and it is obtained that (i) the optimal policy coincides with the optimal policy of this suitable standard control problem, and (ii) the fuzzy optimal value function is of a trapezoidal shape. Two models: a queueing system and a machine replacement problem are provided in order to examplify the theory given.

How to cite

top

Cruz-Suárez, Hugo, Montes-de-Oca, Raúl, and Ortega-Gutiérrez, R. Israel. "An extended version of average Markov decision processes on discrete spaces under fuzzy environment." Kybernetika 59.1 (2023): 160-178. <http://eudml.org/doc/299093>.

@article{Cruz2023,
abstract = {The article presents an extension of the theory of standard Markov decision processes on discrete spaces and with the average cost as the objective function which permits to take into account a fuzzy average cost of a trapezoidal type. In this context, the fuzzy optimal control problem is considered with respect to two cases: the max-order of the fuzzy numbers and the average ranking order of the trapezoidal fuzzy numbers. Each of these cases extends the standard optimal control problem, and for each of them the optimal solution is related to a suitable standard optimal control problem, and it is obtained that (i) the optimal policy coincides with the optimal policy of this suitable standard control problem, and (ii) the fuzzy optimal value function is of a trapezoidal shape. Two models: a queueing system and a machine replacement problem are provided in order to examplify the theory given.},
author = {Cruz-Suárez, Hugo, Montes-de-Oca, Raúl, Ortega-Gutiérrez, R. Israel},
journal = {Kybernetika},
keywords = {Markov decision process; average criterion; trapezoidal fuzzy cost; max-order; average ranking},
language = {eng},
number = {1},
pages = {160-178},
publisher = {Institute of Information Theory and Automation AS CR},
title = {An extended version of average Markov decision processes on discrete spaces under fuzzy environment},
url = {http://eudml.org/doc/299093},
volume = {59},
year = {2023},
}

TY - JOUR
AU - Cruz-Suárez, Hugo
AU - Montes-de-Oca, Raúl
AU - Ortega-Gutiérrez, R. Israel
TI - An extended version of average Markov decision processes on discrete spaces under fuzzy environment
JO - Kybernetika
PY - 2023
PB - Institute of Information Theory and Automation AS CR
VL - 59
IS - 1
SP - 160
EP - 178
AB - The article presents an extension of the theory of standard Markov decision processes on discrete spaces and with the average cost as the objective function which permits to take into account a fuzzy average cost of a trapezoidal type. In this context, the fuzzy optimal control problem is considered with respect to two cases: the max-order of the fuzzy numbers and the average ranking order of the trapezoidal fuzzy numbers. Each of these cases extends the standard optimal control problem, and for each of them the optimal solution is related to a suitable standard optimal control problem, and it is obtained that (i) the optimal policy coincides with the optimal policy of this suitable standard control problem, and (ii) the fuzzy optimal value function is of a trapezoidal shape. Two models: a queueing system and a machine replacement problem are provided in order to examplify the theory given.
LA - eng
KW - Markov decision process; average criterion; trapezoidal fuzzy cost; max-order; average ranking
UR - http://eudml.org/doc/299093
ER -

References

top
  1. Arapostathis, A., Borkar, V. S., Fernández-Gaucherand, E., Gosh, M. K., Marcus, S. I., , SIAM J. Control Optim. 32 (1993), 2, 282-344. MR1205981DOI
  2. Carrero-Vera, K., Cruz-Suárez, H., Montes-de-Oca, R., Discounted Markov decision processes with fuzzy rewards induced by non-fuzzy systems., In: Proc. 10th International Conference on Operations Research and Enterprise Systems ICORES 2021, pp. 49-59. 
  3. Carrero-Vera, K., Cruz-Suárez, H., Montes-de-Oca, R., , Kybernetika 58 (2022), 2, 180-199. MR4467492DOI
  4. Chen, S. J., Chen, S. M., , Appl. Intell. 26 (2007), 1, 1-11. DOI
  5. Chung, Y. L., Tsai, Z. N., A quantized water-filling packet scheduling scheme for downlink transmissions in LTE-advanced systems with carrier aggregation., In: SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks IEEE (2010), pp. 275-279. 
  6. Diamond, P., Kloeden, P., Metric Spaces of Fuzzy Sets: Theory and Applications., World Scientific, Singapore 1994. MR1337027
  7. Ebrahimnejad, A., , Appl. Soft Comput. 19 (2014), 171-176. MR3414360DOI
  8. Furukawa, N., , Optimization 40 (1997), 171-192. MR1620380DOI
  9. Hernández-Lerma, O., Lasserre, J. B., Discrete-Time Markov Control Processes: Basic Optimality Criteria., Springer-Verlag, New York, 1996. Zbl0840.93001MR1363487
  10. Kageyama, M., , J. Comput. Appl. Math. 224 (2009), 1, 140-145. MR2474219DOI
  11. Kaur, A., Kumar, A., , Appl. Soft Comput. 12 (2012), 3, 1201-1213. MR3040892DOI
  12. Konstantin, E., Avrachenkov, E., Sanchez, E., 10.1023/A:1015729400380, Fuzzy Optim. Decis. Making 1 (2002), 12, 143-159. MR1921754DOI10.1023/A:1015729400380
  13. Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y., A fuzzy treatment of uncertain Markov decision processes: average case., In: Proc. ASSM2000 International Conference on Applied Stochastic System Modeling, Kyoto 2000, pp. 148-157. MR1782634
  14. Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y., , Eur. J. Oper. Res. 92 (1996), 3, 649-662. MR1328908DOI
  15. Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y., Markov decision processes with fuzzy rewards., J. Nonlinear Convex Anal. 4 (1996), 1, 105-116. MR1986973
  16. Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y., Perceptive evaluation for the optimal discounted reward in Markov decision processes., In: International Conference on Modeling Decisions for Artificial Intelligence, Springer 2005, pp. 283-293. 
  17. Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y., , Fuzzy Sets and Systems 157 (2006), 19, 2674-2682. MR2328391DOI
  18. López-Díaz, M., Ralescu, D. A., , Comput. Statist. Data Anal. 51 (2006), 109-114. MR2297590DOI
  19. Puri, M. L., Ralescu, D. A., , J. Math. Anal. Appl. 114 (1986), 402-422. MR0833596DOI
  20. Puterman, M., Markov Decision Processes: Discrete Stochastic Dynamic Programming., Wiley, 1994. Zbl1184.90170MR1270015
  21. Rani, D., Gulati, T. R., , J. Transp. Secur. 7 (2014), 3, 277-287. DOI
  22. Rani, D., Gulati, T. R., Kumar, A., , Sadhana 39 (2014),3, 573-581. MR3225832DOI
  23. Rezvani, S., Molani, M., Representation of trapezoidal fuzzy numbers with shape function., Ann. Fuzzy Math. Inform. 8 (2014), 89-112. MR3214770
  24. Ross, S., Applied Probability Models with Optimization Applications., Holden Day, 1996. MR0264792
  25. Semmouri, A., Jourhmane, M., Belhallaj, Z., , Ann. Oper. Res. 295 (2020), 769-786. MR4181708DOI
  26. Sennott, L., Stochastic Dynamic Programming and Control of Queueing Systems., Systems. Wiley, New York 1999. MR1645435
  27. Syropoulos, A., Grammenos, T., A Modern Introduction to Fuzzy Mathematics., Wiley, New Jersey 2020. 
  28. Wang, J., Ma, X., Xu, Z., Zhan, J., , Inform. Sci. 552 (2021), 328-351. MR4197247DOI
  29. Zadeh, L., , Inform. Control 8 (1965), 338-353. Zbl0942.00007MR0219427DOI

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.