This work concerns Markov decision processes with finite state space and compact action sets. The decision maker is supposed to have a constant-risk sensitivity coefficient, and a control policy is graded via the risk-sensitive expected total-reward criterion associated with nonnegative one-step rewards. Assuming that the optimal value function is finite, under mild continuity and compactness restrictions the following result is established: If the number of ergodic classes when a stationary policy...

In the present study the notion of watershed contour dynamics, defined within the framework of mathematical morphology, is examined. It is shown that the dynamics are a direct measure of the “sharpness” of transition between neighboring watershed basins. The expressions for the expected value and the statistical error of the estimation of contour dynamics are derived in the presence of noise, based on extreme value theory. The sensitivity of contour dynamics to noise is studied. A statistical approach...

This article presents a new concept for a statistical fault detection system, including the detection, diagnosis, and prediction of faults. Theoretical material has been collected to provide a complete algorithm making possible the design of a usable system for statistical inference on the basis of the current value of a symptom vector. The use of elements of artificial intelligence enables self-correction and adaptation to changing conditions. The mathematical apparatus is founded on the methodology...

The paper solves the problem of minimization of the Kullback divergence between a partially known and a completely known probability distribution. It considers two probability distributions of a random vector $(u_1,x_1,...,u_T,x_T)$ on a sample space of $2T$ dimensions. One of the distributions is known, the other is known only partially. Namely, only the conditional probability distributions of $x_{\tau }$ given $u_1,x_1,...,u_{\tau -1},x_{\tau -1},u_{\tau }$ are known for $\tau =1,...,T$. Our objective is to determine the remaining conditional probability distributions of $u_{\tau }$ given $u_1,x_1,...,u_{\tau -1},x_{\tau -1}$ such...

A class of finite-dimensional stationary dynamic control systems described by linear stochastic ordinary differential state equations with a single point delay in the state variables is considered. Using a theorem and methods adopted directly from deterministic controllability problems, necessary and sufficient conditions for various kinds of stochastic relative controllability are formulated and proved. It will be demonstrated that under suitable assumptions the relative controllability of an associated...

Finite-dimensional stationary dynamic control systems described by linear stochastic ordinary differential state equations with multiple point delays in control are considered. Using the notation, theorems and methods used for deterministic controllability problems for linear dynamic systems with delays in control as well as necessary and sufficient conditions for various kinds of stochastic relative controllability in a given time interval are formulated and proved. It will be proved that, under...

A zero-sum stochastic differential game problem on infinite horizon with continuous and impulse controls is studied. We obtain the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities. We also obtain a verification theorem which provides an optimal strategy of the game.

A zero-sum stochastic differential game problem on infinite horizon with continuous and impulse controls is studied. We obtain the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities. We also obtain a verification theorem which provides an optimal strategy of the game.

The definition and some existence theorems for stochastic differential inclusions depending only on selections theorems are given.

The definition and some existence theorems for stochastic differential inclusion dZₜ ∈ F(Zₜ)dXₜ, where F and X are set valued stochastic processes, are given.

In this paper we consider stochastic differential equations on Banach spaces (not Hilbert). The system is semilinear and the principal operator generating a C₀-semigroup is perturbed by a class of bounded linear operators considered as feedback operators from an admissible set. We consider the corresponding family of measure valued functions and present sufficient conditions for weak compactness. Then we consider applications of this result to several interesting optimal feedback control problems....

In this paper we consider the question of optimal control for a class of stochastic evolution equations on infinite dimensional Hilbert spaces with controls appearing in both the drift and the diffusion operators. We consider relaxed controls (measure valued random processes) and briefly present some results on the question of existence of mild solutions including their regularity followed by a result on existence of partially observed optimal relaxed controls. Then we develop the necessary conditions...

Inverse problem is a current practice in engineering where the goal is to identify parameters from observed data through numerical models. These numerical models, also called Simulators, are built to represent the phenomenon making possible the inference. However, such representation can include some part of variability or commonly called uncertainty (see [4]), arising from some variables of the model. The phenomenon we study is the fuel mass needed to link two given countries with a commercial...

This paper considers the properties of a minimum variance self-tuning tracker for MIMO systems described by ARMAX models. It is assumed that the stochastic noise has a non-Gaussian distribution. Such an assumption introduces into a recursive algorithm a nonlinear transformation of the prediction error. The system under consideration is minimum phase with different dimensions for input and output vectors. In the paper the concept of Kronecker's product is used, which allows us to represent unknown...