On a mean reward from a Markov replacement process with only one isolated class of recurrent states
On absorption times and Dirichlet eigenvalues
This paper gives a stochastic representation in spectral terms for the absorption time T of a finite Markov chain which is irreducible and reversible outside the absorbing point. This yields quantitative informations on the parameters of a similar representation due to O'Cinneide for general chains admitting real eigenvalues. In the discrete time setting, if the underlying Dirichlet eigenvalues (namely the eigenvalues of the Markov transition operator restricted to the functions vanishing on...
On adaptive control of a partially observed Markov chain
A control problem for a partially observable Markov chain depending on a parameter with long run average cost is studied. Using uniform ergodicity arguments it is shown that, for values of the parameter varying in a compact set, it is possible to consider only a finite number of nearly optimal controls based on the values of actually computable approximate filters. This leads to an algorithm that guarantees nearly selfoptimizing properties without identifiability conditions. The algorithm is based...
On infection spreading and competition between independent random walks.
On limit properties of the reward from a Markov replacement process
On Markovian traffic with applications to TES processes.
On perturbations of strongly admissible prior distributions
On the exchanges between Wolfgang Doeblin and Bohuslav Hostinský
We present the letters sent by Wolfgang Doeblin to Bohuslav Hostinský between 1936 and 1938. They concern some aspects of the general theory of Markov chains and the solutions of the Chapman-Kolmogorov equation that Doeblin was then establishing for his PhD thesis.
On the lengths of excursions of some Markov processes
On the relative lengths of excursions derived from a stable subordinator