A connection between controlled Markov chains and martingales
A consumption-investment problem modelled as a discounted Markov decision process
In this paper a problem of consumption and investment is presented as a model of a discounted Markov decision process with discrete-time. In this problem, it is assumed that the wealth is affected by a production function. This assumption gives the investor a chance to increase his wealth before the investment. For the solution of the problem there is established a suitable version of the Euler Equation (EE) which characterizes its optimal policy completely, that is, there are provided conditions...
A controller and a stopper game with degenerate variance control.
A diffusion approximation in the ruin problem for a controlled Markov chain
A fast algorithm for the two dimensional HJB equation of stochastic control
This paper analyses the implementation of the generalized finite differences method for the HJB equation of stochastic control, introduced by two of the authors in [Bonnans and Zidani, SIAM J. Numer. Anal. 41 (2003) 1008–1021]. The computation of coefficients needs to solve at each point of the grid (and for each control) a linear programming problem. We show here that, for two dimensional problems, this linear programming problem can be solved in operations, where is the size of the stencil....
A fast algorithm for the two dimensional HJB equation of stochastic control
This paper analyses the implementation of the generalized finite differences method for the HJB equation of stochastic control, introduced by two of the authors in [Bonnans and Zidani, SIAM J. Numer. Anal.41 (2003) 1008–1021]. The computation of coefficients needs to solve at each point of the grid (and for each control) a linear programming problem. We show here that, for two dimensional problems, this linear programming problem can be solved in O(pmax) operations, where pmax is the size of...
A general stochastic maximum principle for singular control problems.
A general stochastic target problem with jump diffusion and an application to a hedging problem for large investors.
A generalization of Ueno's inequality for n-step transition probabilities
We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.
A generalized dual maximizer for the Monge–Kantorovich transport problem
The dual attainment of the Monge–Kantorovich transport problem is analyzed in a general setting. The spaces X,Y are assumed to be polish and equipped with Borel probability measures μ and ν. The transport cost function c : X × Y → [0,∞] is assumed to be Borel measurable. We show that a dual optimizer always exists, provided we interpret it as a projective limit of certain finitely additive measures. Our methods are functional analytic and rely on Fenchel’s perturbation technique.
A generalized dual maximizer for the Monge–Kantorovich transport problem∗
The dual attainment of the Monge–Kantorovich transport problem is analyzed in a general setting. The spaces X,Y are assumed to be polish and equipped with Borel probability measures μ and ν. The transport cost function c : X × Y → [0,∞] is assumed to be Borel measurable. We show that a dual optimizer always exists, provided we interpret it as a projective limit of certain finitely additive measures. Our methods are functional analytic...
A linear numerical scheme for nonlinear BSDEs with uniformly continuous coefficients.
A mathematical correction problem
A multidimensional singular stochastic control problem on a finite time horizon
A singular stochastic control problem in n dimensions with timedependent coefficients on a finite time horizon is considered. We show that the value function for this problem is a generalized solution of the corresponding HJB equation with locally bounded second derivatives with respect to the space variables and the first derivative with respect to time. Moreover, we prove that an optimal control exists and is unique
A note on estimation in controlled diffusion processes
A note on the exponential stability of a matrix Riccati equation of stochastic control
A note on the optimal portfolio problem in discrete processes
We deal with the optimal portfolio problem in discrete-time setting. Employing the discrete Itô formula, which is developed by Fujita, we establish the discrete Hamilton–Jacobi–Bellman (d-HJB) equation for the value function. Simple examples of the d-HJB equation are also discussed.
A pension fund in the accumulation phase: a stochastic control approach
In this paper we propose and study a continuous time stochastic model of optimal allocation for a defined contribution pension fund in the accumulation phase. The level of wealth is constrained to stay above a "solvency level". The fund manager can invest in a riskless asset and in a risky asset, but borrowing and short selling are prohibited. The model is naturally formulated as an optimal stochastic control problem with state constraints and is treated by the dynamic programming approach. We show...
A perturbation approach to approximate value iteration for average cost Markov decision processes with Borel spaces and bounded costs
The present paper studies the approximate value iteration (AVI) algorithm for the average cost criterion with bounded costs and Borel spaces. It is shown the convergence of the algorithm and provided a performance bound assuming that the model satisfies a standard continuity-compactness assumption and a uniform ergodicity condition. This is done for the class of approximation procedures that can be represented by linear positive operators which give exact representation of constant functions and...
A polynomial solution to regulation and tracking. II. Stochastic problem