Advanced Search

We introduce average cost optimal adaptive policies in a class of discrete-time Markov control processes with Borel state and action spaces, allowing unbounded costs. The processes evolve according to the system equations $x_{t+1}=F(x_t,a_t,{\xi }_t)$, t=1,2,..., with i.i.d. ${ℝ}^k$-valued random vectors ${\xi }_t$, which are observable but whose density ϱ is unknown.

We consider a class of discrete-time Markov control processes with Borel state and action spaces, and ${\Re }^k$-valued i.i.d. disturbances with unknown density $\rho .$ Supposing possibly unbounded costs, we combine suitable density estimation methods of $\rho$ with approximation procedures of the optimal cost function, to show the existence of a sequence $\left\{{\widehat{f}}_t\right\}$ of minimizers converging to an optimal stationary policy $f_{\infty }.$

We consider semi-Markov control models with Borel state and action spaces, possibly unbounded costs, and holding times with a generalized exponential distribution with unknown mean θ. Assuming that such a distribution does not depend on the state-action pairs, we introduce a Bayesian estimation procedure for θ, which combined with a variant of the vanishing discount factor approach yields average cost optimal policies.

We study the adaptive control problem for discrete-time Markov control processes with Borel state and action spaces and possibly unbounded one-stage costs. The processes are given by recurrent equations $x_{t+1}=F(x_t,a_t,{\xi }_t),t=0,1,...$ with i.i.d. ${\Re }^k$-valued random vectors ${\xi }_t$ whose density $\rho$ is unknown. Assuming observability of ${\xi }_t$ we propose the procedure of statistical estimation of $\rho$ that allows us to prove discounted asymptotic optimality of two types of adaptive policies used early for the processes with bounded costs.

This work deals with a class of discrete-time zero-sum Markov games whose state process $\left\{x_t\right\}$ evolves according to the equation $x_{t+1}=F(x_t,a_t,b_t,{\xi }_t),$ where $a_t$ and $b_t$ represent the actions of player 1 and 2, respectively, and $\left\{{\xi }_t\right\}$ is a sequence of independent and identically distributed random variables with unknown distribution $\theta$. Assuming possibly unbounded payoff, and using the empirical distribution to estimate $\theta$, we introduce approximation schemes for the value of the game as well as for optimal strategies considering both,...

We are concerned with a class of $GI/GI/1$ queueing systems with controlled service rates, in which the waiting times are only observed when they take zero value. Applying a suitable filtering process, we show the existence of optimal control policies under a discounted optimality criterion.

The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process $\left\{x_t\right\}$ and the discount process $\left\{{\alpha }_t\right\}$ evolve according to the coupled difference equations $x_{t+1}=F(x_t,{\alpha }_t,a_t,{\xi }_t),$ ${\alpha }_{t+1}=G({\alpha }_t,{\eta }_t)$ where the state and discount disturbance processes $\left\{{\xi }_t\right\}$ and $\left\{{\eta }_t\right\}$ are sequences of i.i.d. random variables with densities ${\rho }^{\xi }$ and ${\rho }^{\eta }$ respectively. The main objective is to introduce approximation algorithms of the optimal...

This paper deals with a class of partially observable discounted Markov decision processes defined on Borel state and action spaces, under unbounded one-stage cost. The discount rate is a stochastic process evolving according to a difference equation, which is also assumed to be partially observable. Introducing a suitable control model and filtering processes, we prove the existence of optimal control policies. In addition, we illustrate our results in a class of GI/GI/1 queueing systems where...