Advanced Search

This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.

This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.

Let $(X_n,n\ge 1),({\tilde{X}}_n,n\ge 1)$ be two sequences of i.i.d. random vectors with values in ${ℝ}^k$ and $S_n=X_1+\cdots +X_n$, ${\tilde{S}}_n={\tilde{X}}_1+\cdots +{\tilde{X}}_n$, $n\ge 1$. Assuming that $E{X}_1=E{\tilde{X}}_1$, $E|X_1{|}^2<\infty$, $E|{\tilde{X}}_1{|}^{k+2}<\infty$ and the existence of a density of ${\tilde{X}}_1$ satisfying the certain conditions we prove the following inequalities: $$v(S_n,{\tilde{S}}_n)\le cmax\{v(X_1,{\tilde{X}}_1),{\zeta }_2(X_1,{\tilde{X}}_1)\},n=1,2,\cdots ,$$ where $v$ and ${\zeta }_2$ are the total variation and Zolotarev’s metrics, respectively.

The upper bounds of the uniform distance $\rho \left({\sum }_{k=1}^{\nu }{X}_k,{\sum }_{k=1}^{\nu }{\tilde{X}}_k\right)$ between two sums of a random number $\nu$ of independent random variables are given. The application of these bounds is illustrated by stability (continuity) estimating in models in queueing and risk theory.

For a discrete-time Markov control process with the transition probability $p$, we compare the total discounted costs $V_{\beta }$ $\left({\pi }_{\beta }\right)$ and $V_{\beta }\left({\tilde{\pi }}_{\beta }\right)$, when applying the optimal control policy ${\pi }_{\beta }$ and its approximation ${\tilde{\pi }}_{\beta }$. The policy ${\tilde{\pi }}_{\beta }$ is optimal for an approximating process with the transition probability $\tilde{p}$. A cost per stage for considered processes can be unbounded. Under certain ergodicity assumptions we establish the upper bound for the relative stability index $[V_{\beta }\left({\tilde{\pi }}_{\beta }\right)-V_{\beta }\left({\pi }_{\beta }\right)]/V_{\beta }\left({\pi }_{\beta }\right)$. This bound does not depend on a discount...

We deal with a stochastic programming problem that can be inconsistent. To overcome the inconsistency we apply Tikhonov's regularization technique, and, using recent results on the convergence rate of empirical measures in Wasserstein metric, we treat the following two related problems: 1. A choice of regularization parameters that guarantees the convergence of the minimization procedure. 2. Estimation of the rate of convergence in probability. Considering both light and heavy tail distributions...

We study the adaptive control problem for discrete-time Markov control processes with Borel state and action spaces and possibly unbounded one-stage costs. The processes are given by recurrent equations $x_{t+1}=F(x_t,a_t,{\xi }_t),t=0,1,...$ with i.i.d. ${\Re }^k$-valued random vectors ${\xi }_t$ whose density $\rho$ is unknown. Assuming observability of ${\xi }_t$ we propose the procedure of statistical estimation of $\rho$ that allows us to prove discounted asymptotic optimality of two types of adaptive policies used early for the processes with bounded costs.

We study the limit behavior of certain classes of dependent random sequences (processes) which do not possess the Markov property. Assuming these processes depend on a control parameter we show that the optimization of the control can be reduced to a problem of nonlinear optimization. Under certain hypotheses we establish the stability of such optimization problems.

We study the stability of the classical optimal sequential probability ratio test based on independent identically distributed observations $X_1,X_2,\cdots$ when testing two simple hypotheses about their common density $f$: $f=f_0$ versus $f=f_1$. As a functional to be minimized, it is used a weighted sum of the average (under $f_0$) sample number and the two types error probabilities. We prove that the problem is reduced to stopping time optimization for a ratio process generated by $X_1,X_2,\cdots$ with the density $f_0$. For ${\tau }_{*}$ being the corresponding...