The search session has expired. Please query the service again.
The search session has expired. Please query the service again.
The search session has expired. Please query the service again.
The search session has expired. Please query the service again.
In this note we focus attention on identifying optimal policies and on elimination suboptimal policies minimizing optimality criteria in discrete-time Markov decision processes with finite state space and compact action set. We present unified approach to value iteration algorithms that enables to generate lower and upper bounds on optimal values, as well as on the current policy. Using the modified value iterations it is possible to eliminate suboptimal actions and to identify an optimal policy...
We consider an infinite system of hard balls in undergoing Brownian motions
and submitted to a smooth pair potential.
It is modelized by an infinite-dimensional stochastic differential equation
with an infinite-dimensional local time term.
Existence and uniqueness of a strong solution is proven for such an equation
with fixed deterministic initial condition. We also show
that Gibbs measures are reversible measures.
This second part of our two part work on i.d. process has four main goals:(1) To develop a potential operator for recurrent i.d. (infinitely divisible) processes and to use this operator to find the asymptotic behavior of the hitting distribution and Green’s function for relatively compact sets in the recurrent case.(2) To develop the appropriate notion of an equilibrium measure and Robin’s constant for Borel sets.(3) To establish the asymptotic behavior questions of a potential theoretic nature...
We show that associated with every i.d. (infinitely divisible) process on a locally compact, non-compact 2nd countable Abelian group is a corresponding potential theory that yields definitive results on the behavior of the process in both space and time. Our results are general, no density or other smoothness assumptions are made on the process. In this first part of two part work we have four main goals.(1) To lay the probabilistic foundation of such processes. This mainly consists in giving the...
We study the infinitesimal generators of evolutions of linear mappings on the space of polynomials, which correspond to a special class of Markov processes with polynomial regressions called quadratic harnesses. We relate the infinitesimal generator to the unique solution of a certain commutation equation, and we use the commutation equation to find an explicit formula for the infinitesimal generator of free quadratic harnesses.
The article considers the effectiveness of various methods used to solve systems of linear equations (which emerge while modeling computer networks and systems with Markov chains) and the practical influence of the methods applied on accuracy. The paper considers some hybrids of both direct and iterative methods. Two varieties of the Gauss elimination will be considered as an example of direct methods: the LU factorization method and the WZ factorization method. The Gauss-Seidel iterative method...
Sensitivity analysis of irreducible Markov chains considers an original Markov chain with transition probability matrix and modified Markov chain
with transition probability matrix . For their respective stationary probability vectors ,
some of the following charactristics are usually studied: for asymptotical stability [3], for componentwise stability or sensitivity [1]. For functional transition probabilities, and stationary probability vector , derivatives are also used for studying...
Este trabajo presenta diversas extensiones de la identidad de Wald, con interpretaciones en términos del comportamiento de un embalse. Se considera la independencia y diversos casos de dependencia (markoviana homogénea, markoviana no homogénea) de las variables aleatorias "entrada neta" al embalse. En tiempo continuo, se incluye una identidad de Wald para el proceso de Poisson compuesto.
Currently displaying 1 –
20 of
66