Displaying similar documents to “A note on Markov operators and transition systems”

On the exchanges between Wolfgang Doeblin and Bohuslav Hostinský

Laurent Mazliak (2007)

Revue d'histoire des mathématiques

Similarity:

We present the letters sent by Wolfgang Doeblin to Bohuslav Hostinský between 1936 and 1938. They concern some aspects of the general theory of Markov chains and the solutions of the Chapman-Kolmogorov equation that Doeblin was then establishing for his PhD thesis.

Markov's property of the Cantor ternary set

Leokadia Białas, Alexander Volberg (1993)

Studia Mathematica

Similarity:

We prove that the Cantor ternary set E satisfies the classical Markov inequality (see [Ma]): for each polynomial p of degree at most n (n = 0, 1, 2,...) (M) | p ' ( x ) | ≤ M n m s u p E | p | for x ∈ E, where M and m are positive constants depending only on E.

Markov operators acting on Polish spaces

Tomasz Szarek (1997)

Annales Polonici Mathematici

Similarity:

We prove a new sufficient condition for the asymptotic stability of Markov operators acting on measures. This criterion is applied to iterated function systems.

Estimates for perturbations of average Markov decision processes with a minimal state and upper bounded by stochastically ordered Markov chains

Raúl Montes-de-Oca, Francisco Salem-Silva (2005)

Kybernetika

Similarity:

This paper deals with Markov decision processes (MDPs) with real state space for which its minimum is attained, and that are upper bounded by (uncontrolled) stochastically ordered (SO) Markov chains. We consider MDPs with (possibly) unbounded costs, and to evaluate the quality of each policy, we use the objective function known as the average cost. For this objective function we consider two Markov control models ℙ and ℙ 1 . ℙ and ℙ 1 have the same components except for the transition laws....

The uniqueness of invariant measures for Markov operators

Tomasz Szarek (2008)

Studia Mathematica

Similarity:

It is shown that Markov operators with equicontinuous dual operators which overlap supports have at most one invariant measure. In this way we extend the well known result proved for Markov operators with the strong Feller property by R. Z. Khas'minski.

Conditional Markov chains - construction and properties

Tomasz R. Bielecki, Jacek Jakubowski, Mariusz Niewęgłowski (2015)

Banach Center Publications

Similarity:

In this paper we study finite state conditional Markov chains (CMCs). We give two examples of CMCs, one which admits intensity, and another one, which does not admit an intensity. We also give a sufficient condition under which a doubly stochastic Markov chain is a CMC. In addition we provide a method for construction of conditional Markov chains via change of measure.