Time-discretization for controlled Markov processes. I. General approximation results
Time-discretization for controlled Markov processes. II. A jump and diffusion application
Time-varying Markov decision processes with state-action-dependent discount factors and unbounded costs
In this paper we are concerned with a class of time-varying discounted Markov decision models with unbounded costs and state-action dependent discount factors. Specifically we study controlled systems whose state process evolves according to the equation , with state-action dependent discount factors of the form , where and are the control and the random disturbance at time , respectively. Assuming that the sequences of functions , and converge, in certain sense, to , and , our...
Topological dual of with application to stochastic systems on Hilbert space
In this paper, we prove that the topological dual of the Banach space of bounded measurable functions with values in the space of nuclear operators, furnished with the natural topology, is isometrically isomorphic to the space of finitely additive linear operator-valued measures having bounded variation in a Banach space containing the space of bounded linear operators. This is then applied to a stochastic structural control problem. An optimal operator-valued measure, considered as the structural...
Tracking problem for linear periodic, discrete-time stochastic systems in Hilbert spaces.
Transient phenomena and self-optimizing control of Markov chains
Two-level stochastic control for a linear system with nonclassical information
A problem of control law design for large scale stochastic systems is discussed. Nonclassical information pattern is considered. A two-level hierarchical control structure with a coordinator on the upper level and local controllers on the lower level is proposed. A suboptimal algorithm with a partial decomposition of calculations and decentralized local control is obtained. A simple example is presented to illustrate the proposed approach.
Un problema di controllo stocastico per lo sfruttamento di una risorsa rinnovabile
Un problème de contrôle géométrique et les équations de Hamilton-Jacobi-Bellman
Unicité des lois optimales du contrôle stochastique continu dans le cas complètement observable
Uniqueness and approximate computation of optimal incomplete transportation plans
For α∈(0, 1) an α-trimming, P∗, of a probability P is a new probability obtained by re-weighting the probability of any Borel set, B, according to a positive weight function, f≤1/(1−α), in the way P∗(B)=∫Bf(x)P(dx). If P, Q are probability measures on euclidean space, we consider the problem of obtaining the best L2-Wasserstein approximation between: (a) a fixed probability and trimmed versions of the other; (b) trimmed versions of both probabilities. These best trimmed approximations naturally...
Uniqueness of optimal policies as a generic property of discounted Markov decision processes: Ekeland's variational principle approach
Many examples in optimization, ranging from Linear Programming to Markov Decision Processes (MDPs), present more than one optimal solution. The study of this non-uniqueness is of great mathematical interest. In this paper the authors show that in a specific family of discounted MDPs, non-uniqueness is a “fragile” property through Ekeland's Principle for each problem with at least two optimal policies; a perturbed model is produced with a unique optimal policy. This result not only supersedes previous...
Viability, invariance and reachability for controlled piecewise deterministic Markov processes associated to gene networks
We aim at characterizing viability, invariance and some reachability properties of controlled piecewise deterministic Markov processes (PDMPs). Using analytical methods from the theory of viscosity solutions, we establish criteria for viability and invariance in terms of the first order normal cone. We also investigate reachability of arbitrary open sets. The method is based on viscosity techniques and duality for some associated linearized problem. The theoretical results are applied to general...
Viability, invariance and reachability for controlled piecewise deterministic Markov processes associated to gene networks
We aim at characterizing viability, invariance and some reachability properties of controlled piecewise deterministic Markov processes (PDMPs). Using analytical methods from the theory of viscosity solutions, we establish criteria for viability and invariance in terms of the first order normal cone. We also investigate reachability of arbitrary open sets. The method is based on viscosity techniques and duality for some associated linearized problem. The theoretical results are applied to general...
Viability, invariance and reachability for controlled piecewise deterministic Markov processes associated to gene networks
We aim at characterizing viability, invariance and some reachability properties of controlled piecewise deterministic Markov processes (PDMPs). Using analytical methods from the theory of viscosity solutions, we establish criteria for viability and invariance in terms of the first order normal cone. We also investigate reachability of arbitrary open sets. The method is based on viscosity techniques and duality for some associated linearized problem. The theoretical results are applied to general...
Viscosity solutions of two classes of coupled Hamilton-Jacobi-Bellman equations.
Zajištění fyzikální realizovatelnosti optimální -parametrové diskrétní lineární regulace řešené ve Wienerově smyslu
К задаче Дирихле для уравнения Беллмана в плоской области
Некоторые новые результаты из теории управляемых диффузионных процессов
О задаче Дирихле для уравнения Беллмана в плоской области