Combining -dependence with markovness
Commande optimale hiérarchisée d'une population de chaînes de Markov
Comportement asymptotique d’une classe de chaînes de Markov sur
Compositions of random functions on a finite set.
Computing the Bounds on the Loss Rates
Computing the reliability of systems with statistical dependent elements.
Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games
In this paper we present the extraproximal method for computing the Stackelberg/Nash equilibria in a class of ergodic controlled finite Markov chains games. We exemplify the original game formulation in terms of coupled nonlinear programming problems implementing the Lagrange principle. In addition, Tikhonov's regularization method is employed to ensure the convergence of the cost-functions to a Stackelberg/Nash equilibrium point. Then, we transform the problem into a system of equations in the...
Conditioning properties of the stationary distribution for a Markov chain.
Convergence issues in the theory and practice of iterative aggregation/disaggregation methods.
Convergence of iterates of a transfer operator, application to dynamical systems and to Markov chains
We present a spectral theory for a class of operators satisfying a weak “Doeblin–Fortet” condition and apply it to a class of transition operators. This gives the convergence of the series , , under some regularity assumptions and implies the central limit theorem with a rate in for the corresponding Markov chain. An application to a non uniformly hyperbolic transformation on the interval is also given.
Convergence of iterates of a transfer operator, application to dynamical systems and to Markov chains
We present a spectral theory for a class of operators satisfying a weak “Doeblin–Fortet" condition and apply it to a class of transition operators. This gives the convergence of the series ∑k≥0krPkƒ, , under some regularity assumptions and implies the central limit theorem with a rate in for the corresponding Markov chain. An application to a non uniformly hyperbolic transformation on the interval is also given.
Convergent Nonnegative Matrices and Iterative Methods for Consistent Linear Systems.
Cutoff for samples of Markov chains
Cutoff for samples of Markov chains
We study the convergence to equilibrium of n-samples of independent Markov chains in discrete and continuous time. They are defined as Markov chains on the n-fold Cartesian product of the initial state space by itself, and they converge to the direct product of n copies of the initial stationary distribution. Sharp estimates for the convergence speed are given in terms of the spectrum of the initial chain. A cutoff phenomenon occurs in the sense that as n tends to infinity, the total variation...
Cycle time of stochastic max-plus linear systems.
Deterministic optimal policies for Markov control processes with pathwise constraints
This paper deals with discrete-time Markov control processes in Borel spaces with unbounded rewards. Under suitable hypotheses, we show that a randomized stationary policy is optimal for a certain expected constrained problem (ECP) if and only if it is optimal for the corresponding pathwise constrained problem (pathwise CP). Moreover, we show that a certain parametric family of unconstrained optimality equations yields convergence properties that lead to an approximation scheme which allows us to...
Die Klassifikation der Zustände bei inhomogenen Markoffschen Ketten.
Dimension and Entropy of a Non-ergodic Markovian Process and its Relation to Rademacher Riesz Products.
Directed forests with application to algorithms related to Markov chains
This paper is devoted to computational problems related to Markov chains (MC) on a finite state space. We present formulas and bounds for characteristics of MCs using directed forest expansions given by the Matrix Tree Theorem. These results are applied to analysis of direct methods for solving systems of linear equations, aggregation algorithms for nearly completely decomposable MCs and the Markov chain Monte Carlo procedures.
Discrete version of Dungey’s proof for the gradient heat kernel estimate on coverings
We obtain another proof of a Gaussian upper estimate for a gradient of the heat kernel on cofinite covering graphs whose covering transformation group has a polynomial volume growth. It is proved by using the temporal regularity of the discrete heat kernel obtained by Blunck [2] and Christ [3] along with the arguments of Dungey [7] on covering manifolds.