On a mean reward from a Markov replacement process with only one isolated class of recurrent states
On additive and multiplicative (controlled) Poisson equations
Assuming that a Markov process satisfies the minorization property, existence and properties of the solutions to the additive and multiplicative Poisson equations are studied using splitting techniques. The problem is then extended to the study of risk sensitive and risk neutral control problems and corresponding Bellman equations.
On convergence in distribution of the Markov chain generated by the filter kernel induced by a fully dominated Hidden Markov Model [Book]
On dependence structure of copula-based Markov chains
We consider dependence coefficients for stationary Markov chains. We emphasize on some equivalencies for reversible Markov chains. We improve some known results and provide a necessary condition for Markov chains based on Archimedean copulas to be exponential ρ-mixing. We analyse the example of the Mardia and Frechet copula families using small sets.
On deterministic approximation of Markov processes by ordinary differential equations.
On impulsive control with long run average cost criterion
On invariant measures of nonlinear Markov processes.
On the adaptive control of countable Markov chains
On the asymptotic variance in the central limit theorem for particle filters
Particle filter algorithms approximate a sequence of distributions by a sequence of empirical measures generated by a population of simulated particles. In the context of Hidden Markov Models (HMM), they provide approximations of the distribution of optimal filters associated to these models. For a given set of observations, the behaviour of particle filters, as the number of particles tends to infinity, is asymptotically Gaussian, and the asymptotic variance in the central limit theorem depends...
On the asymptotic variance in the central limit theorem for particle filters
Particle filter algorithms approximate a sequence of distributions by a sequence of empirical measures generated by a population of simulated particles. In the context of Hidden Markov Models (HMM), they provide approximations of the distribution of optimal filters associated to these models. For a given set of observations, the behaviour of particle filters, as the number of particles tends to infinity, is asymptotically Gaussian, and the asymptotic variance in the central limit theorem depends...
On the classification of Markov chains via occupation measures
We consider a Markov chain on a locally compact separable metric space and with a unique invariant probability. We show that such a chain can be classified into two categories according to the type of convergence of the expected occupation measures. Several properties in each category are investigated.
On the existence of invariant densities for Markov operators
On the Kantorovich-Rubinstein maximum principle for the Fortet-Mourier norm
A new version of the maximum principle is presented. The classical Kantorovich-Rubinstein principle gives necessary conditions for the maxima of a linear functional acting on the space of Lipschitzian functions. The maximum value of this functional defines the Hutchinson metric on the space of probability measures. We show an analogous result for the Fortet-Mourier metric. This principle is then applied in the stability theory of Markov-Feller semigroups.
On the limit distributions of kth order statistics for semi-pareto processes
Asymptotic properties of the kth largest values for semi-Pareto processes are investigated. Conditions for convergence in distribution of the kth largest values are given. The obtained limit laws are represented in terms of a compound Poisson distribution.
On the Markov chain central limit theorem.
On the mean speed of convergence of empirical and occupation measures in Wasserstein distance
In this work, we provide non-asymptotic bounds for the average speed of convergence of the empirical measure in the law of large numbers, in Wasserstein distance. We also consider occupation measures of ergodic Markov chains. One motivation is the approximation of a probability measure by finitely supported measures (the quantization problem). It is found that rates for empirical or occupation measures match or are close to previously known optimal quantization rates in several cases. This is notably...
On the probabilistic multichain Poisson equation
This paper introduces necessary and/or sufficient conditions for the existence of solutions (g,h) to the probabilistic multichain Poisson equation (a) g = Pg and (b) g+h-Ph = f, with a given charge f, where P is a Markov kernel (or transition probability function) on a general measurable space. The existence conditions are derived via three different approaches, using (1) canonical pairs, (2) Cesàro averages, and (3) resolvents.
On the rate of convergence in the weak invariance principle for dependent random variables with applications to Markov chains
We prove an invariance principle for non-stationary random processes and establish a rate of convergence under a new type of mixing condition. The dependence is exponentially decaying in the gap between the past and the future and is controlled by an assumption on the characteristic function of the finite-dimensional increments of the process. The distinctive feature of the new mixing condition is that the dependence increases exponentially in the dimension of the increments. The proposed mixing...
On the spectral analysis of second-order Markov chains
Second order Markov chains which are trajectorially reversible are considered. Contrary to the reversibility notion for usual Markov chains, no symmetry property can be deduced for the corresponding transition operators. Nevertheless and even if they are not diagonalizable in general, we study some features of their spectral decompositions and in particular the behavior of the spectral gap under appropriate perturbations is investigated. Our quantitative and qualitative results confirm that the...
On the Structure of Spatial Branching Processes
The paper is a contribution to the theory of branching processes with discrete time and a general phase space in the sense of [2]. We characterize the class of regular, i.e. in a sense sufficiently random, branching processes (Φk) k∈Z by almost sure properties of their realizations without making any assumptions about stationarity or existence of moments. This enables us to classify the clans of (Φk) into the regular part and the completely non-regular part. It turns out that the completely non-regular branching...