All a b c d e f g h i j k l m n o p q r s t u v w x y z Other

Previous Page 7 Next

Displaying 121 – 140 of 363

Showing per page

Growth rates and average optimality in risk-sensitive Markov decision chains

Karel Sladký (2008)

Kybernetika

In this note we focus attention on characterizations of policies maximizing growth rate of expected utility, along with average of the associated certainty equivalent, in risk-sensitive Markov decision chains with finite state and action spaces. In contrast to the existing literature the problem is handled by methods of stochastic dynamic programming on condition that the transition probabilities are replaced by general nonnegative matrices. Using the block-triangular decomposition of a collection...

(Homogeneous) markovian bridges

Vincent Vigon (2011)

Annales de l'I.H.P. Probabilités et statistiques

(Homogeneous) Markov bridges are (time homogeneous) Markov chains which begin at a given point and end at a given point. The price to pay for preserving the homogeneity is to work with processes with a random life-span. Bridges are studied both for themselves and for their use in describing the transformations of Markov chains: restriction on a random interval, time reversal, time change, various conditionings comprising the confinement in some part of the state space. These bridges lead us to look...

Identification of optimal policies in Markov decision processes

Karel Sladký (2010)

Kybernetika

In this note we focus attention on identifying optimal policies and on elimination suboptimal policies minimizing optimality criteria in discrete-time Markov decision processes with finite state space and compact action set. We present unified approach to value iteration algorithms that enables to generate lower and upper bounds on optimal values, as well as on the current policy. Using the modified value iterations it is possible to eliminate suboptimal actions and to identify an optimal policy...

Influence of preconditioning and blocking on accuracy in solving Markovian models

Beata Bylina, Jarosław Bylina (2009)

International Journal of Applied Mathematics and Computer Science

The article considers the effectiveness of various methods used to solve systems of linear equations (which emerge while modeling computer networks and systems with Markov chains) and the practical influence of the methods applied on accuracy. The paper considers some hybrids of both direct and iterative methods. Two varieties of the Gauss elimination will be considered as an example of direct methods: the LU factorization method and the WZ factorization method. The Gauss-Seidel iterative method...

Insensitivity analysis of Markov chains

Kocurek, Martin (2010)

Programs and Algorithms of Numerical Mathematics

Sensitivity analysis of irreducible Markov chains considers an original Markov chain with transition probability matrix P and modified Markov chain with transition probability matrix P . For their respective stationary probability vectors π , π ˜ , some of the following charactristics are usually studied: ∥ π - π ˜ ∥ p for asymptotical stability [3], | π i - π ˜ i | , | π i - π ˜ i | π i for componentwise stability or sensitivity [1]. For functional transition probabilities, P = P ( t ) and stationary probability vector π ( t ) , derivatives are also used for studying...

Instante de primer vaciado y extensiones de la identidad de Wald.

Guillermo Domínguez Oliván, Miguel San Miguel Marco (1989)

Trabajos de Estadística

Este trabajo presenta diversas extensiones de la identidad de Wald, con interpretaciones en términos del comportamiento de un embalse. Se considera la independencia y diversos casos de dependencia (markoviana homogénea, markoviana no homogénea) de las variables aleatorias "entrada neta" al embalse. En tiempo continuo, se incluye una identidad de Wald para el proceso de Poisson compuesto.

Irreducible Markov systems on Polish spaces

Katarzyna Horbacz, Tomasz Szarek (2006)

Studia Mathematica

Contractive Markov systems on Polish spaces which arise from graph directed constructions of iterated function systems with place dependent probabilities are considered. It is shown that their stability may be studied using the concentrating methods developed by the second author [Dissert. Math. 415 (2003)]. In this way Werner's results obtained in a locally compact case [J. London Math. Soc. 71 (2005)] are extended to a noncompact setting.

Currently displaying 121 – 140 of 363