This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.

The work treats the problem of fault detection for processes described by partial differential equations as that of maximizing the power of a parametric hypothesis test which checks whether or not system parameters have nominal values. A simple node activation strategy is discussed for the design of a sensor network deployed in a spatial domain that is supposed to be used while detecting changes in the underlying parameters which govern the process evolution. The setting considered relates to a...

In this paper we solve the basic fractional analogue of the classical linear-quadratic Gaussian regulator problem in continuous-time with partial observation. For a controlled linear system where both the state and observation processes are driven by fractional Brownian motions, we describe explicitly the optimal control policy which minimizes a quadratic performance criterion. Actually, we show that a separation principle holds, i.e., the optimal control separates into two stages based on optimal...

This paper studies a class of discrete-time discounted semi-Markov control model on Borel spaces. We assume possibly unbounded costs and a non-stationary exponential form in the discount factor which depends of on a rate, called the discount rate. Given an initial discount rate the evolution in next steps depends on both the previous discount rate and the sojourn time of the system at the current state. The new results provided here are the existence and the approximation of optimal policies for...

A production inventory problem with limited backlogging and with stockouts is described in a discrete time, stochastic optimal control framework with finite horizon. It is proved by dynamic programming methods that an optimal policy is of (s,S)-type. This means that in every period the policy is completely determined by two fixed levels of the stochastic inventory process considered.

Let $L$ be a parabolic second order differential operator on the domain $\overline{\Pi }=\left[0,T\right]\times ℝ.$ Given a function $\widehat{u}:ℝ\to R$ and $\widehat{x}\>0$ such that the support of $\widehat{u}$ is contained in $(-\infty ,-\widehat{x}]$, we let $\widehat{y}:\overline{\Pi }\to ℝ$ be the solution to the equation:$$L\widehat{y}=0,\widehat{y}{|}_{\left\{0\right\}\times ℝ}=\widehat{u}.$$Given positive bounds $0\<x_0\<x_1,$ we seek a function $u$ with support in $\left[x_0,x_1\right]$ such that the corresponding solution $y$ satisfies:$$y(t,0)=\widehat{y}(t,0)\forall t\in \left[0,T\right].$$We prove in this article that, under some regularity conditions on the coefficients of $L,$ continuous solutions are unique and dense in the sense that $\widehat{y}{|}_{[0,T]\times \left\{0\right\}}$ can be $C^0$-approximated, but an exact solution does not...

Let L be a parabolic second order differential operator on the domain $\overline{\Pi }=\left[0,T\right]\times ℝ.$ Given a function $\widehat{u}:ℝ\to R$ and $\widehat{x}>0$ such that the support of û is contained in $(-\infty ,-\widehat{x}]$, we let $\widehat{y}:\overline{\Pi }\to ℝ$ be the solution to the equation: $$L\widehat{y}=0,\widehat{y}{|}_{\left\{0\right\}\times ℝ}=\widehat{u}.$$ Given positive bounds $0<x_0<x_1,$ we seek a function u with support in $\left[x_0,x_1\right]$ such that the corresponding solution y satisfies: $$y(t,0)=\widehat{y}(t,0)\forall t\in \left[0,T\right].$$ We prove in this article that, under some regularity conditions on the coefficients of L, continuous solutions are unique and dense in the sense that $\widehat{y}{|}_{[0,T]\times \left\{0\right\}}$ can be C0-approximated, but an exact solution...

This work concerns Markov decision processes with finite state space and compact action sets. The decision maker is supposed to have a constant-risk sensitivity coefficient, and a control policy is graded via the risk-sensitive expected total-reward criterion associated with nonnegative one-step rewards. Assuming that the optimal value function is finite, under mild continuity and compactness restrictions the following result is established: If the number of ergodic classes when a stationary policy...

The paper solves the problem of minimization of the Kullback divergence between a partially known and a completely known probability distribution. It considers two probability distributions of a random vector $(u_1,x_1,...,u_T,x_T)$ on a sample space of $2T$ dimensions. One of the distributions is known, the other is known only partially. Namely, only the conditional probability distributions of $x_{\tau }$ given $u_1,x_1,...,u_{\tau -1},x_{\tau -1},u_{\tau }$ are known for $\tau =1,...,T$. Our objective is to determine the remaining conditional probability distributions of $u_{\tau }$ given $u_1,x_1,...,u_{\tau -1},x_{\tau -1}$ such...

A zero-sum stochastic differential game problem on infinite horizon with continuous and impulse controls is studied. We obtain the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities. We also obtain a verification theorem which provides an optimal strategy of the game.

A zero-sum stochastic differential game problem on infinite horizon with continuous and impulse controls is studied. We obtain the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities. We also obtain a verification theorem which provides an optimal strategy of the game.