Advanced Search

For a discrete-time Markov control process with the transition probability $p$, we compare the total discounted costs $V_{\beta }$ $\left({\pi }_{\beta }\right)$ and $V_{\beta }\left({\tilde{\pi }}_{\beta }\right)$, when applying the optimal control policy ${\pi }_{\beta }$ and its approximation ${\tilde{\pi }}_{\beta }$. The policy ${\tilde{\pi }}_{\beta }$ is optimal for an approximating process with the transition probability $\tilde{p}$. A cost per stage for considered processes can be unbounded. Under certain ergodicity assumptions we establish the upper bound for the relative stability index $[V_{\beta }\left({\tilde{\pi }}_{\beta }\right)-V_{\beta }\left({\pi }_{\beta }\right)]/V_{\beta }\left({\pi }_{\beta }\right)$. This bound does not depend on a discount...

We extend previous results of the same authors ([11]) on the effects of perturbation in the transition probability of a Markov cost chain for discounted Markov control processes. Supposing valid, for each stationary policy, conditions of Lyapunov and Harris type, we get upper bounds for the index of perturbations, defined as the difference of the total expected discounted costs for the original Markov control process and the perturbed one. We present examples that satisfy our conditions.

We analyse a Markov chain and perturbations of the transition probability and the one-step cost function (possibly unbounded) defined on it. Under certain conditions, of Lyapunov and Harris type, we obtain new estimates of the effects of such perturbations via an index of perturbations, defined as the difference of the total expected discounted costs between the original Markov chain and the perturbed one. We provide an example which illustrates our analysis.

This paper deals with Markov decision processes (MDPs) with real state space for which its minimum is attained, and that are upper bounded by (uncontrolled) stochastically ordered (SO) Markov chains. We consider MDPs with (possibly) unbounded costs, and to evaluate the quality of each policy, we use the objective function known as the average cost. For this objective function we consider two Markov control models $\mathbb{P}$ and ${\mathbb{P}}_1$. $\mathbb{P}$ and ${\mathbb{P}}_1$ have the same components except for the transition laws. The transition...