Fatou's lemma and Lebesgue's convergence theorem for measures.
Invariant probabilities for Feller-Markov chains.
Existence of average optimal policies in Markov control processes with strictly unbounded costs
Approximation and adaptive control of Markov processes: Average reward criterion
Semi-Markov control models with average costs
This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model.
On the classification of Markov chains via occupation measures
We consider a Markov chain on a locally compact separable metric space and with a unique invariant probability. We show that such a chain can be classified into two categories according to the type of convergence of the expected occupation measures. Several properties in each category are investigated.
Average cost Markov control processes with weighted norms: existence of canonical policies
This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.
Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality
We consider discrete-time Markov control processes on Borel spaces and infinite-horizon undiscounted cost criteria which are sensitive to the growth rate of finite-horizon costs. These criteria include, at one extreme, the grossly underselective average cost
Average cost Markov control processes with weighted norms: value iteration
This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.
On the probabilistic multichain Poisson equation
This paper introduces necessary and/or sufficient conditions for the existence of solutions (g,h) to the probabilistic multichain Poisson equation (a) g = Pg and (b) g+h-Ph = f, with a given charge f, where P is a Markov kernel (or transition probability function) on a general measurable space. The existence conditions are derived via three different approaches, using (1) canonical pairs, (2) Cesàro averages, and (3) resolvents.
Limiting average cost control problems in a class of discrete-time stochastic systems
We consider a class of -valued stochastic control systems, with possibly unbounded costs. The systems evolve according to a discrete-time equation (t = 0,1,... ), for each fixed n = 0,1,..., where the are i.i.d. random vectors, and the Gₙ are given functions converging pointwise to some function as n → ∞. Under suitable hypotheses, our main results state the existence of stationary control policies that are expected average cost (EAC) optimal and sample path average cost (SPAC) optimal for...
The linear programming approach to deterministic optimal control problems
Given a deterministic optimal control problem (OCP) with value function, say , we introduce a linear program and its dual whose values satisfy . Then we give conditions under which (i) there is no duality gap
Deterministic optimal policies for Markov control processes with pathwise constraints
This paper deals with discrete-time Markov control processes in Borel spaces with unbounded rewards. Under suitable hypotheses, we show that a randomized stationary policy is optimal for a certain expected constrained problem (ECP) if and only if it is optimal for the corresponding pathwise constrained problem (pathwise CP). Moreover, we show that a certain parametric family of unconstrained optimality equations yields convergence properties that lead to an approximation scheme which allows us to...
Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria
Page 1