Two kinds of strategies for a multiarmed Markov bandit problem with controlled arms are considered: a strategy with forcing and a strategy with randomization. The choice of arm and control function in both cases is based on the current value of the average cost per unit time functional. Some simulation results are also presented.
We establish necessary and sufficient conditions of near-optimality for nonlinear systems governed by forward-backward stochastic differential equations with controlled jump processes (FBSDEJs in short). The set of controls under consideration is necessarily convex. The proof of our result is based on Ekeland's variational principle and continuity in some sense of the state and adjoint processes with respect to the control variable. We prove that under an additional hypothesis, the near-maximum...
The author investigates non ergodic versions of several well known limit theorems for strictly stationary processes. In some cases, the assumptions which are given with respect to general invariant measure, guarantee the validity of the theorem with respect to ergodic components of the measure. In other cases, the limit theorem can fail for all ergodic components, while for the original invariant measure it holds.
In this paper we study a general concept of nonuniform exponential dichotomy in mean square for stochastic skew-evolution semiflows in Hilbert spaces. We obtain a variant for the stochastic case of some well-known results, of the deterministic case, due to R. Datko: Uniform asymptotic stability of evolutionary processes in a Banach space, SIAM J. Math. Anal., 3(1972), 428–445. Our approach is based on the extension of some techniques used in the deterministic case for the study of asymptotic behavior...
We prove that under some assumptions a one-dimensional Itô equation has a strong solution concentrated on a finite spatial interval, and the pathwise uniqueness holds.
A system with a single activated unit which can be in states is considered. Inspections of the system are carried out at discrete time instants. The process of deterioration of the unit is supposed to be Markovian. The unit by its operation brings an income which is monotonically dependent on its state. A replacement of the unit is associated with certain costs. The paper gives an effective algorithm for finding the replacement strategy maximizing the average income of the system per unit time....