Decision making in an incompletely known stochastic system, II
Decision making in an incompletely known stochastic system, I
Decomposition of large-scale stochastic optimal control problems
In this paper, we present an Uzawa-based heuristic that is adapted to certain type of stochastic optimal control problems. More precisely, we consider dynamical systems that can be divided into small-scale subsystems linked through a static almost sure coupling constraint at each time step. This type of problem is common in production/portfolio management where subsystems are, for instance, power units, and one has to supply a stochastic power demand at each time step. We outline the framework...
Degenerate variance control in the one-dimensional stationary case.
Deterministic Markov Nash equilibria for potential discrete-time stochastic games
In this paper, we study the problem of finding deterministic (also known as feedback or closed-loop) Markov Nash equilibria for a class of discrete-time stochastic games. In order to establish our results, we develop a potential game approach based on the dynamic programming technique. The identified potential stochastic games have Borel state and action spaces and possibly unbounded nondifferentiable cost-per-stage functions. In particular, the team (or coordination) stochastic games and the stochastic...
Deterministic optimal policies for Markov control processes with pathwise constraints
This paper deals with discrete-time Markov control processes in Borel spaces with unbounded rewards. Under suitable hypotheses, we show that a randomized stationary policy is optimal for a certain expected constrained problem (ECP) if and only if it is optimal for the corresponding pathwise constrained problem (pathwise CP). Moreover, we show that a certain parametric family of unconstrained optimality equations yields convergence properties that lead to an approximation scheme which allows us to...
Differential games of partial information forward-backward doubly SDE and applications
This paper addresses a new differential game problem with forward-backward doubly stochastic differential equations. There are two distinguishing features. One is that our game systems are initial coupled, rather than terminal coupled. The other is that the admissible control is required to be adapted to a subset of the information generated by the underlying Brownian motions. We establish a necessary condition and a sufficient condition for an equilibrium point of nonzero-sum games and a saddle...
Diffusion approximation for a controlled service system
Discounted Markov control processes induced by deterministic systems
This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function,...
Discrete time arbitrage under transaction costs
Conditions for the absence of arbitrage in discrete time markets with various kinds of transaction costs are shown.
Discrete time infinite horizon risk sensitive portfolio selection with proportional transaction costs
Long run risk sensitive portfolio selection is considered with proportional transaction costs. In the paper two methods to prove existence of solutions to suitable Bellman equations are presented. The first method is based on discounted cost approximation and requires uniform absolute continuity of iterations of transition operators of the factor process. The second method is based on uniform ergodicity of portions of the capital invested in assets and requires additional assumptions concerning...
Discrete time risk sensitive portfolio optimization with consumption and proportional transaction costs
Risk sensitive and risk neutral long run portfolio problems with consumption and proportional transaction costs are studied. Existence of solutions to suitable Bellman equations is shown. The asymptotics of the risk sensitive cost when the risk factor converges to 0 is then considered. It turns out that optimal strategies are stationary functions of the portfolio (portions of the wealth invested in assets) and of economic factors. Furthermore an optimal portfolio strategy for a risk neutral control...
Discrete-time Markov control processes with discounted unbounded costs: Optimality criteria
Discrete-time Markov control processes with recursive discount rates
This work analyzes a discrete-time Markov Control Model (MCM) on Borel spaces when the performance index is the expected total discounted cost. This criterion admits unbounded costs. It is assumed that the discount rate in any period is obtained by using recursive functions and a known initial discount rate. The classic dynamic programming method for finite-horizon case is verified. Under slight conditions, the existence of deterministic non-stationary optimal policies for infinite-horizon case...
Double-stepped adaptive control for hybrid systems with unknown Markov jumps and stochastic noises
This paper is concerned with the sampled-data based adaptive linear quadratic (LQ) control of hybrid systems with both unmeasurable Markov jump processes and stochastic noises. By the least matching error estimation algorithm, parameter estimates are presented. By a double-step (DS) sampling approach and the certainty equivalence principle, a sampled-data based adaptive LQ control is designed. The DS-approach is characterized by a comparatively large estimation step for parameter estimation and...
Double-stepped adaptive control for hybrid systems with unknown Markov jumps and stochastic noises
This paper is concerned with the sampled-data based adaptive linear quadratic (LQ) control of hybrid systems with both unmeasurable Markov jump processes and stochastic noises. By the least matching error estimation algorithm, parameter estimates are presented. By a double-step (DS) sampling approach and the certainty equivalence principle, a sampled-data based adaptive LQ control is designed. The DS-approach is characterized by a comparatively large estimation step for parameter estimation and...
Doubly reflected BSDEs with call protection and their approximation
We study the numerical approximation of doubly reflected backward stochastic differential equations with intermittent upper barrier (RIBSDEs). These denote reflected BSDEs in which the upper barrier is only active on certain random time intervals. From the point of view of financial interpretation, RIBSDEs arise as pricing equations of game options with constrained callability. In a Markovian set-up we prove a convergence rate for a time-discretization scheme by simulation to an RIBSDE. We also...
Dual response systems optimization. An approximation approach.
Dynamic programming for an investment/consumption problem in illiquid markets with regime-switching
We consider an illiquid financial market with different regimes modeled by a continuous time finite-state Markov chain. The investor can trade a stock only at the discrete arrival times of a Cox process with intensity depending on the market regime. Moreover, the risky asset price is subject to liquidity shocks, which change its rate of return and volatility, and induce jumps on its dynamics. In this setting, we study the problem of an economic agent optimizing her expected utility from consumption...
Dynamic programming for stochastic target problems and geometric flows