We study the existence of sample path average cost (SPAC-) optimal policies for Markov control processes on Borel spaces with strictly unbounded costs, i.e., costs that grow without bound on the complement of compact subsets. Assuming only that the cost function is lower semicontinuous and that the transition law is weakly continuous, we show the existence of a relaxed policy with 'minimal' expected average cost and that the optimal average cost is the limit of discounted programs. Moreover, we...

We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria.

The article is devoted to Markov reward chains in discrete-time setting with finite state spaces. Unfortunately, the usual optimization criteria examined in the literature on Markov decision chains, such as a total discounted, total reward up to reaching some specific state (called the first passage models) or mean (average) reward optimality, may be quite insufficient to characterize the problem from the point of a decision maker. To this end it seems that it may be preferable if not necessary...

This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.

The work treats the problem of fault detection for processes described by partial differential equations as that of maximizing the power of a parametric hypothesis test which checks whether or not system parameters have nominal values. A simple node activation strategy is discussed for the design of a sensor network deployed in a spatial domain that is supposed to be used while detecting changes in the underlying parameters which govern the process evolution. The setting considered relates to a...

In this paper we solve the basic fractional analogue of the classical linear-quadratic Gaussian regulator problem in continuous-time with partial observation. For a controlled linear system where both the state and observation processes are driven by fractional Brownian motions, we describe explicitly the optimal control policy which minimizes a quadratic performance criterion. Actually, we show that a separation principle holds, i.e., the optimal control separates into two stages based on optimal...

This paper studies a class of discrete-time discounted semi-Markov control model on Borel spaces. We assume possibly unbounded costs and a non-stationary exponential form in the discount factor which depends of on a rate, called the discount rate. Given an initial discount rate the evolution in next steps depends on both the previous discount rate and the sojourn time of the system at the current state. The new results provided here are the existence and the approximation of optimal policies for...

Let $L$ be a parabolic second order differential operator on the domain $\overline{\Pi }=\left[0,T\right]\times ℝ.$ Given a function $\widehat{u}:ℝ\to R$ and $\widehat{x}\>0$ such that the support of $\widehat{u}$ is contained in $(-\infty ,-\widehat{x}]$, we let $\widehat{y}:\overline{\Pi }\to ℝ$ be the solution to the equation:$$L\widehat{y}=0,\widehat{y}{|}_{\left\{0\right\}\times ℝ}=\widehat{u}.$$Given positive bounds $0\<x_0\<x_1,$ we seek a function $u$ with support in $\left[x_0,x_1\right]$ such that the corresponding solution $y$ satisfies:$$y(t,0)=\widehat{y}(t,0)\forall t\in \left[0,T\right].$$We prove in this article that, under some regularity conditions on the coefficients of $L,$ continuous solutions are unique and dense in the sense that $\widehat{y}{|}_{[0,T]\times \left\{0\right\}}$ can be $C^0$-approximated, but an exact solution does not...

Let L be a parabolic second order differential operator on the domain $\overline{\Pi }=\left[0,T\right]\times ℝ.$ Given a function $\widehat{u}:ℝ\to R$ and $\widehat{x}>0$ such that the support of û is contained in $(-\infty ,-\widehat{x}]$, we let $\widehat{y}:\overline{\Pi }\to ℝ$ be the solution to the equation: $$L\widehat{y}=0,\widehat{y}{|}_{\left\{0\right\}\times ℝ}=\widehat{u}.$$ Given positive bounds $0<x_0<x_1,$ we seek a function u with support in $\left[x_0,x_1\right]$ such that the corresponding solution y satisfies: $$y(t,0)=\widehat{y}(t,0)\forall t\in \left[0,T\right].$$ We prove in this article that, under some regularity conditions on the coefficients of L, continuous solutions are unique and dense in the sense that $\widehat{y}{|}_{[0,T]\times \left\{0\right\}}$ can be C0-approximated, but an exact solution...