Displaying similar documents to “A perturbation approach to approximate value iteration for average cost Markov decision processes with Borel spaces and bounded costs”

Second Order optimality in Markov decision chains

Karel Sladký (2017)

Kybernetika

Similarity:

The article is devoted to Markov reward chains in discrete-time setting with finite state spaces. Unfortunately, the usual optimization criteria examined in the literature on Markov decision chains, such as a total discounted, total reward up to reaching some specific state (called the first passage models) or mean (average) reward optimality, may be quite insufficient to characterize the problem from the point of a decision maker. To this end it seems that it may be preferable if not...

Two algorithms based on Markov chains and their application to recognition of protein coding genes in prokaryotic genomes

Małgorzata Grabińska, Paweł Błażej, Paweł Mackiewicz (2013)

Applicationes Mathematicae

Similarity:

Methods based on the theory of Markov chains are most commonly used in the recognition of protein coding sequences. However, they require big learning sets to fill up all elements in transition probability matrices describing dependence between nucleotides in the analyzed sequences. Moreover, gene prediction is strongly influenced by the nucleotide bias measured by e.g. G+C content. In this paper we compare two methods: (i) the classical GeneMark algorithm, which uses a three-periodic...

Identification of optimal policies in Markov decision processes

Karel Sladký (2010)

Kybernetika

Similarity:

In this note we focus attention on identifying optimal policies and on elimination suboptimal policies minimizing optimality criteria in discrete-time Markov decision processes with finite state space and compact action set. We present unified approach to value iteration algorithms that enables to generate lower and upper bounds on optimal values, as well as on the current policy. Using the modified value iterations it is possible to eliminate suboptimal actions and to identify an optimal...

An algorithm for a new method of change-point analysis in the independent Poisson sequence

Chihiro Hirotsu, Harukazu Tsuruta (2017)

Biometrical Letters

Similarity:

Step change-point and slope change-point models in the independent Poisson sequence are developed based on accumulated and doubly-accumulated statistics. The method for the step change-point model developed in Section 2 is an alternative to the likelihood ratio test of Worsley (1986) and the algorithm for p-value calculation based on the first-order Markov property is the same as that given there. Different algorithms for the non-null distribution and inference on the change-point itself...

Semi-Markov control models with average costs

Fernando Luque-Vásquez, Onésimo Hernández-Lerma (1999)

Applicationes Mathematicae

Similarity:

This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.

Risk probability optimization problem for finite horizon continuous time Markov decision processes with loss rate

Haifeng Huo, Xian Wen (2021)

Kybernetika

Similarity:

This paper presents a study the risk probability optimality for finite horizon continuous-time Markov decision process with loss rate and unbounded transition rates. Under drift condition, which is slightly weaker than the regular condition, as detailed in existing literature on the risk probability optimality Semi-Markov decision processes, we prove that the value function is the unique solution of the corresponding optimality equation, and demonstrate the existence of a risk probability...