Discrete-time Markov control processes with recursive discount rates

Yofre H. García; Juan González-Hernández

Displaying similar documents to “Discrete-time Markov control processes with recursive discount rates”

Solutions of semi-Markov control models with recursive discount rates and approximation by $ϵ$ -optimal policies

Yofre H. García, Juan González-Hernández (2019)

Kybernetika

Similarity:

This paper studies a class of discrete-time discounted semi-Markov control model on Borel spaces. We assume possibly unbounded costs and a non-stationary exponential form in the discount factor which depends of on a rate, called the discount rate. Given an initial discount rate the evolution in next steps depends on both the previous discount rate and the sojourn time of the system at the current state. The new results provided here are the existence and the approximation of optimal...

Time-discretization for controlled Markov processes. I. General approximation results

Nico M. van Dijk, Arie Hordijk (1996)

Kybernetika

Similarity:

Estimates of stability of Markov control processes with unbounded costs

Evgueni I. Gordienko, Francisco Salem-Silva (2000)

Kybernetika

Similarity:

For a discrete-time Markov control process with the transition probability $p$ , we compare the total discounted costs $V_{β}$ $(π_{β})$ and $V_{β} ({\tilde{π}}_{β})$ , when applying the optimal control policy $π_{β}$ and its approximation ${\tilde{π}}_{β}$ . The policy ${\tilde{π}}_{β}$ is optimal for an approximating process with the transition probability $\tilde{p}$ . A cost per stage for considered processes can be unbounded. Under certain ergodicity assumptions we establish the upper bound for the relative stability index $[V_{β} ({\tilde{π}}_{β}) - V_{β} (π_{β})] / V_{β} (π_{β})$ . This bound does not depend...

Time-discretization for controlled Markov processes. II. A jump and diffusion application

Nico M. van Dijk, Arie Hordijk (1996)

Kybernetika

Similarity:

Deterministic optimal policies for Markov control processes with pathwise constraints

Armando F. Mendoza-Pérez, Onésimo Hernández-Lerma (2012)

Applicationes Mathematicae

Similarity:

This paper deals with discrete-time Markov control processes in Borel spaces with unbounded rewards. Under suitable hypotheses, we show that a randomized stationary policy is optimal for a certain expected constrained problem (ECP) if and only if it is optimal for the corresponding pathwise constrained problem (pathwise CP). Moreover, we show that a certain parametric family of unconstrained optimality equations yields convergence properties that lead to an approximation scheme which...

Modelling and optimal control of networked systems with stochastic communication protocols

Chaoqun Zhu, Bin Yang, Xiang Zhu (2020)

Kybernetika

Similarity:

This paper is concerned with the finite and infinite horizon optimal control issue for a class of networked control systems with stochastic communication protocols. Due to the limitation of networked bandwidth, only the limited number of sensors and actuators are allowed to get access to network mediums according to stochastic access protocols. A discrete-time Markov chain with a known transition probability matrix is employed to describe the scheduling behaviors of the stochastic access...

Approximate evaluation of continuous review $(R, Q)$ policies in two-echelon inventory systems with stochastic transportation times

Abdullah S. Karaman (2017)

Kybernetika

Similarity:

This paper considers a distribution inventory system that consists of a single warehouse and several retailers. Customer demand arrives at the retailers according to a continuous-time renewal process. Material flow between echelons is driven by reorder point/order quantity inventory control policies. Our objective in this setting is to calculate the long-run inventory, backorder and customer service levels. The challenge in this system is to characterize the demand arrival process at...

The exponential cost optimality for finite horizon semi-Markov decision processes

Haifeng Huo, Xian Wen (2022)

Kybernetika

Similarity:

This paper considers an exponential cost optimality problem for finite horizon semi-Markov decision processes (SMDPs). The objective is to calculate an optimal policy with minimal exponential costs over the full set of policies in a finite horizon. First, under the standard regular and compact-continuity conditions, we establish the optimality equation, prove that the value function is the unique solution of the optimality equation and the existence of an optimal policy by using the...