Limiting average cost control problems in a class of discrete-time stochastic systems

Nadine Hilgert; Onesimo Hernández-Lerma

Displaying similar documents to “Limiting average cost control problems in a class of discrete-time stochastic systems”

Synthesis of optimal control for nonlinear third order systems

Wiesław Szwiec

Similarity:

The paper is devoted to a certain problem in the theory of synthesis of optimal control. The fundamental results of research in the synthesis of optimal control for second order systems are given in [1], [2], [5], [10]. In particular, in [2] and [5] some specific nonlinear systems are investigated, namely the systems related with the differential equations of the forms̈ + F(s,ṡ) = u or s̈ + F(s,ṡ,u) = 0where u is a control parameter in the interval [—1,1]. Some results concerning third...

Time-varying Markov decision processes with state-action-dependent discount factors and unbounded costs

Beatris A. Escobedo-Trujillo, Carmen G. Higuera-Chan (2019)

Kybernetika

Similarity:

In this paper we are concerned with a class of time-varying discounted Markov decision models $ℳ_{n}$ with unbounded costs $c_{n}$ and state-action dependent discount factors. Specifically we study controlled systems whose state process evolves according to the equation $x_{n + 1} = G_{n} (x_{n}, a_{n}, ξ_{n}), n = 0, 1, ...$ , with state-action dependent discount factors of the form $α_{n} (x_{n}, a_{n})$ , where $a_{n}$ and $ξ_{n}$ are the control and the random disturbance at time $n$ , respectively. Assuming that the sequences of functions ${α_{n}}$ , ${c_{n}}$ and ${G_{n}}$ converge, in certain sense, to $α_{\infty}$ ,...

Stability estimating in optimal stopping problem

Elena Zaitseva (2008)

Kybernetika

Similarity:

We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space $X$ . It is supposed that an unknown transition probability $p (\cdot | x)$ , $x \in X$ , is approximated by the transition probability $\tilde{p} (\cdot | x)$ , $x \in X$ , and the stopping rule ${\tilde{τ}}_{*}$ , optimal for $\tilde{p}$ , is applied to the process governed by $p$ . We found an upper bound for the difference between the total expected cost, resulting when applying ${\tilde{τ}}_{*}$ , and the minimal total expected cost. The bound given is a constant times ${sup}_{x \in X} ∥ p (\cdot | x) - \tilde{p} (\cdot | x) ∥$ , where $∥ \cdot ∥$ is the...

Linearization techniques for $𝕃^{\infty}$ See PDF-control problems and dynamic programming principles in classical and $𝕃^{\infty}$ See PDF-control problems

Dan Goreac, Oana-Silvia Serea (2012)

ESAIM: Control, Optimisation and Calculus of Variations

Similarity:

The aim of the paper is to provide a linearization approach to the $𝕃^{\infty}$ See PDF-control problems. We begin by proving a semigroup-type behaviour of the set of constraints appearing in the linearized formulation of (standard) control problems. As a byproduct we obtain a linear formulation of the dynamic programming principle. Then, we use the $𝕃^{p}$ See PDF approach and the associated linear formulations. This seems to be the most appropriate tool for treating $𝕃^{\infty}$ See PDF problems in continuous and...

Systems of Bellman Equations to Stochastic Differential Games with Discount Control

Alain Bensoussan, Jens Frehse (2008)

Bollettino dell'Unione Matematica Italiana

Similarity:

We consider two dimensional diagonal elliptic systems $\Delta u+au=H(x,u,\nabla u)$ which arise from stochastic differential games with discount control. The Hamiltonians $H$ have quadratic growth in $\nabla u$ and a special structure which has notyet been covered by regularity theory. Without smallness condition on $H$ , the existence of a regular solution is established.

A continuous mapping theorem for the argmin-set functional with applications to convex stochastic processes

Dietmar Ferger (2021)

Kybernetika

Similarity:

For lower-semicontinuous and convex stochastic processes $Z_{n}$ and nonnegative random variables $ϵ_{n}$ we investigate the pertaining random sets $A (Z_{n}, ϵ_{n})$ of all $ϵ_{n}$ -approximating minimizers of $Z_{n}$ . It is shown that, if the finite dimensional distributions of the $Z_{n}$ converge to some $Z$ and if the $ϵ_{n}$ converge in probability to some constant $c$ , then the $A (Z_{n}, ϵ_{n})$ converge in distribution to $A (Z, c)$ in the hyperspace of Vietoris. As a simple corollary we obtain an extension of several argmin-theorems in the literature. In particular,...

Soft local times and decoupling of random interlacements

Serguei Popov, Augusto Teixeira (2015)

Journal of the European Mathematical Society

Similarity:

In this paper we establish a decoupling feature of the random interlacement process $ℐ^{u} \subset ℤ^{d}$ at level $u$ , $d \geq 3$ . Roughly speaking, we show that observations of $ℐ^{u}$ restricted to two disjoint subsets $A_{1}$ and $A_{2}$ of $ℤ^{d}$ are approximately independent, once we add a sprinkling to the process $ℐ^{u}$ by slightly increasing the parameter $u$ . Our results differ from previous ones in that we allow the mutual distance between the sets $A_{1}$ and $A_{2}$ to be much smaller than their diameters. We then provide an important application...

On reliability analysis of consecutive $k$ -out-of- $n$ systems with arbitrarily dependent components

Ebrahim Salehi (2016)

Applications of Mathematics

Similarity:

In this paper, we consider the linear and circular consecutive $k$ -out-of- $n$ systems consisting of arbitrarily dependent components. Under the condition that at least $n - r + 1$ components ( $r \leq n$ ) of the system are working at time $t$ , we study the reliability properties of the residual lifetime of such systems. Also, we present some stochastic ordering properties of residual lifetime of consecutive $k$ -out-of- $n$ systems. In the following, we investigate the inactivity time of the component with lifetime...

Optimality of the range for which equivalence between certain measures of smoothness holds

Z. Ditzian (2010)

Studia Mathematica

Similarity:

Recently it was proved for 1 < p < ∞ that $ω^{m} {(f, t)}_{p}$ , a modulus of smoothness on the unit sphere, and $K ̃ ₘ {(f, t^{m})}_{p}$ , a K-functional involving the Laplace-Beltrami operator, are equivalent. It will be shown that the range 1 < p < ∞ is optimal; that is, the equivalence $ω^{m} {(f, t)}_{p} \approx K ̃ ₘ {(f, t^{r})}_{p}$ does not hold either for p = ∞ or for p = 1.

Transportation flow problems with Radon measure variables

Marcus Wagner (2000)

Discussiones Mathematicae, Differential Inclusions, Control and Optimization

Similarity:

For a multidimensional control problem ${(P)}_{K}$ involving controls $u \in L_{\infty}$ , we construct a dual problem ${(D)}_{K}$ in which the variables ν to be paired with u are taken from the measure space rca (Ω,) instead of $(L_{\infty}) *$ . For this purpose, we add to ${(P)}_{K}$ a Baire class restriction for the representatives of the controls u. As main results, we prove a strong duality theorem and saddle-point conditions.

Tykhonov well-posedness of a heat transfer problem with unilateral constraints

Mircea Sofonea, Domingo A. Tarzia (2022)

Applications of Mathematics

Similarity:

We consider an elliptic boundary value problem with unilateral constraints and subdifferential boundary conditions. The problem describes the heat transfer in a domain $D \subset ℝ^{d}$ and its weak formulation is in the form of a hemivariational inequality for the temperature field, denoted by $𝒫$ . We associate to Problem $𝒫$ an optimal control problem, denoted by $𝒬$ . Then, using appropriate Tykhonov triples, governed by a nonlinear operator $G$ and a convex $\tilde{K}$ , we provide results concerning the well-posedness...

On pathwise uniqueness for stochastic differential equations driven by stable Lévy processes

Nicolas Fournier (2013)

Annales de l'I.H.P. Probabilités et statistiques

Similarity:

We study a one-dimensional stochastic differential equation driven by a stable Lévy process of order $α$ with drift and diffusion coefficients $b$ , $σ$ . When $α \in (1, 2)$ , we investigate pathwise uniqueness for this equation. When $α \in (0, 1)$ , we study another stochastic differential equation, which is equivalent in law, but for which pathwise uniqueness holds under much weaker conditions. We obtain various results, depending on whether $α \in (0, 1)$ or $α \in (1, 2)$ and on whether the driving stable process is symmetric or not. Our...

Covariance structure of wide-sense Markov processes of order k ≥ 1

Arkadiusz Kasprzyk, Władysław Szczotka (2006)

Applicationes Mathematicae

Similarity:

A notion of a wide-sense Markov process $X_{t}$ of order k ≥ 1, $X_{t} \sim W M (k)$ , is introduced as a direct generalization of Doob’s notion of wide-sense Markov process (of order k=1 in our terminology). A base for investigation of the covariance structure of $X_{t}$ is the k-dimensional process $x_{t} = (X_{t - k + 1}, . . ., X_{t})$ . The covariance structure of $X_{t} \sim W M (k)$ is considered in the general case and in the periodic case. In the general case it is shown that $X_{t} \sim W M (k)$ iff $x_{t}$ is a k-dimensional WM(1) process and iff the covariance function of $x_{t}$ has the triangular...

Optimal Constants in Khintchine Type Inequalities for Fermions, Rademachers and q-Gaussian Operators

Artur Buchholz (2005)

Bulletin of the Polish Academy of Sciences. Mathematics

Similarity:

For $(P_{k})$ being Rademacher, Fermion or q-Gaussian (-1 ≤ q ≤ 0) operators, we find the optimal constants $C_{2 n}$ , n∈ ℕ, in the inequality $∥ \sum_{k = 1}^{N} A_{k} \otimes P_{k} ∥_{2 n} \leq {[C_{2 n}]}^{1 / 2 n} m a x {∥ (\sum_{k = 1}^{N} A *_{k} A_{k}}^{1 / 2} ∥_{L_{2 n}}, ∥ (\sum_{k = 1}^{N} A_{k} A *_{k}$ 1/2∥L2n $,$ valid for all finite sequences of operators $(A_{k})$ in the non-commutative $L_{2 n}$ space related to a semifinite von Neumann algebra with trace. In particular, $C_{2 n} = (2 n r - 1)!!$ for the Rademacher and Fermion sequences.

Nonconventional limit theorems in averaging

Yuri Kifer (2014)

Annales de l'I.H.P. Probabilités et statistiques

Similarity:

We consider “nonconventional” averaging setup in the form $\frac{d X^{ε} (t)}{d t} = ε B (X^{ε} (t)$ , $𝛯 (q_{1} (t)), 𝛯 (q_{2} (t)), ..., 𝛯 (q_{ℓ} (t)))$ where $𝛯 (t)$ , $t \geq 0$ is either a stochastic process or a dynamical system with sufficiently fast mixing while $q_{j} (t) = α_{j} t$ , $α_{1} l t; α_{2} l t; \dots l t; α_{k}$ and $q_{j}$ , $j = k + 1, ..., ℓ$ grow faster than linearly. We show that the properly normalized error term in the “nonconventional” averaging principle is asymptotically Gaussian.

Giant component and vacant set for random walk on a discrete torus

Itai Benjamini, Alain-Sol Sznitman (2008)

Journal of the European Mathematical Society

Similarity:

We consider random walk on a discrete torus $E$ of side-length $N$ , in sufficiently high dimension $d$ . We investigate the percolative properties of the vacant set corresponding to the collection of sites which have not been visited by the walk up to time $u N^{d}$ . We show that when $u$ is chosen small, as $N$ tends to infinity, there is with overwhelming probability a unique connected component in the vacant set which contains segments of length const $log N$ . Moreover, this connected component occupies a...