The moving average (MA) chart, the exponentially weighted moving average (EWMA) chart and the cumulative sum (CUSUM) chart are the most popular schemes for detecting shifts in a relevant process parameter. Any control chart system of span $k$ is specified by a partition of the space ${ℝ}^k$ into three disjoint parts. We call this partition as the control chart frame of span $k.$ A shift in the process parameter is signalled at time $t$ by having the vector of the last $k$ sample characteristics fall out of the...

This work concerns a discrete-time Markov chain with time-invariant transition mechanism and denumerable state space, which is endowed with a nonnegative cost function with finite support. The performance of the chain is measured by the (long-run) risk-sensitive average cost and, assuming that the state space is communicating, the existence of a solution to the risk-sensitive Poisson equation is established, a result that holds even for transient chains. Also, a sufficient criterion ensuring that...

The problem of minimizing the ergodic or time-averaged cost for a controlled diffusion with partial observations can be recast as an equivalent control problem for the associated nonlinear filter. In analogy with the completely observed case, one may seek the value function for this problem as the vanishing discount limit of value functions for the associated discounted cost problems. This passage is justified here for the scalar case under a stability hypothesis, leading in particular to a "martingale"...

In this paper we are concerned with a class of time-varying discounted Markov decision models ${ℳ}_n$ with unbounded costs $c_n$ and state-action dependent discount factors. Specifically we study controlled systems whose state process evolves according to the equation $x_{n+1}=G_n(x_n,a_n,{\xi }_n),n=0,1,...$, with state-action dependent discount factors of the form ${\alpha }_n(x_n,a_n)$, where $a_n$ and ${\xi }_n$ are the control and the random disturbance at time $n$, respectively. Assuming that the sequences of functions $\left\{{\alpha }_n\right\}$,$\left\{c_n\right\}$ and $\left\{G_n\right\}$ converge, in certain sense, to ${\alpha }_{\infty }$, $c_{\infty }$ and $G_{\infty }$, our...

In this paper, we prove that the topological dual of the Banach space of bounded measurable functions with values in the space of nuclear operators, furnished with the natural topology, is isometrically isomorphic to the space of finitely additive linear operator-valued measures having bounded variation in a Banach space containing the space of bounded linear operators. This is then applied to a stochastic structural control problem. An optimal operator-valued measure, considered as the structural...

A problem of control law design for large scale stochastic systems is discussed. Nonclassical information pattern is considered. A two-level hierarchical control structure with a coordinator on the upper level and local controllers on the lower level is proposed. A suboptimal algorithm with a partial decomposition of calculations and decentralized local control is obtained. A simple example is presented to illustrate the proposed approach.