Hamilton-Jacobi-Bellman equations for the optimal control of a state equation with memory

Guillaume Carlier; Rabah Tahraoui

ESAIM: Control, Optimisation and Calculus of Variations (2010)

  • Volume: 16, Issue: 3, page 744-763
  • ISSN: 1292-8119

Abstract

top
This article is devoted to the optimal control of state equations with memory of the form: x ˙ ( t ) = F ( x ( t ) , u ( t ) , ∫ 0 + ∞ A ( s ) x ( t - s ) d s ) , t > 0 , with initial conditions x ( 0 ) = x , x ( - s ) = z ( s ) , s > 0 . Denoting by y x , z , u the solution of the previous Cauchy problem and: v ( x , z ) : = inf u ∈ V { ∫ 0 + ∞ e - λ s L ( y x , z , u ( s ) , u ( s ) ) d s } where V is a class of admissible controls, we prove that v is the only viscosity solution of an Hamilton-Jacobi-Bellman equation of the form: λ v ( x , z ) + H ( x , z , ∇ x v ( x , z ) ) + 〈 D z v ( x , z ) , z ˙ 〉 = 0 in the sense of the theory of viscosity solutions in infinite-dimensions of Crandall and Lions.

How to cite

top

Carlier, Guillaume, and Tahraoui, Rabah. "Hamilton-Jacobi-Bellman equations for the optimal control of a state equation with memory." ESAIM: Control, Optimisation and Calculus of Variations 16.3 (2010): 744-763. <http://eudml.org/doc/250723>.

@article{Carlier2010,
abstract = { This article is devoted to the optimal control of state equations with memory of the form: $\dot\{x\}(t)=F(x(t),u(t), \int_0^\{+\infty\} A(s) x(t-s) \{\rm d\}s), \; t>0,$with initial conditions $x(0)=x, \; x(-s)=z(s), s>0$. Denoting by $y_\{x, z, u\}$ the solution of the previous Cauchy problem and:$ v(x,z):=\inf_\{u\in V\} \lbrace \int_0^\{+\infty\} \{\rm e\}^\{-\lambda s \} L(y_\{x,z,u\}(s), u(s))\{\rm d\}s\rbrace $ where V is a class of admissible controls, we prove that v is the only viscosity solution of an Hamilton-Jacobi-Bellman equation of the form:$ \lambda v(x,z)+H(x,z,\nabla_x v(x,z))+\langle D_z v(x,z), \dot\{z\} \rangle=0 $ in the sense of the theory of viscosity solutions in infinite-dimensions of Crandall and Lions. },
author = {Carlier, Guillaume, Tahraoui, Rabah},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
keywords = {Dynamic programming; state equations with memory; viscosity solutions; Hamilton-Jacobi-Bellman equations in infinite dimensions; dynamic programming},
language = {eng},
month = {7},
number = {3},
pages = {744-763},
publisher = {EDP Sciences},
title = {Hamilton-Jacobi-Bellman equations for the optimal control of a state equation with memory},
url = {http://eudml.org/doc/250723},
volume = {16},
year = {2010},
}

TY - JOUR
AU - Carlier, Guillaume
AU - Tahraoui, Rabah
TI - Hamilton-Jacobi-Bellman equations for the optimal control of a state equation with memory
JO - ESAIM: Control, Optimisation and Calculus of Variations
DA - 2010/7//
PB - EDP Sciences
VL - 16
IS - 3
SP - 744
EP - 763
AB - This article is devoted to the optimal control of state equations with memory of the form: $\dot{x}(t)=F(x(t),u(t), \int_0^{+\infty} A(s) x(t-s) {\rm d}s), \; t>0,$with initial conditions $x(0)=x, \; x(-s)=z(s), s>0$. Denoting by $y_{x, z, u}$ the solution of the previous Cauchy problem and:$ v(x,z):=\inf_{u\in V} \lbrace \int_0^{+\infty} {\rm e}^{-\lambda s } L(y_{x,z,u}(s), u(s)){\rm d}s\rbrace $ where V is a class of admissible controls, we prove that v is the only viscosity solution of an Hamilton-Jacobi-Bellman equation of the form:$ \lambda v(x,z)+H(x,z,\nabla_x v(x,z))+\langle D_z v(x,z), \dot{z} \rangle=0 $ in the sense of the theory of viscosity solutions in infinite-dimensions of Crandall and Lions.
LA - eng
KW - Dynamic programming; state equations with memory; viscosity solutions; Hamilton-Jacobi-Bellman equations in infinite dimensions; dynamic programming
UR - http://eudml.org/doc/250723
ER -

References

top
  1. C.T.H. Baker, G.A. Bocharov and F.A. Rihan, A Report on the Use of Delay Differential Equations in Numerical Modelling in the Biosciences. Technical report, Manchester Centre for Computational Mathematics, UK (1999).  
  2. A. Bensoussan, G. Da Prato, M. Delfour and S.K. Mitter, Representation and control of infinite dimensional systems. Second Edition, Birkhäuser (2007).  
  3. M. Bardi and I.C. Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997).  
  4. G. Barles, Solutions de viscosité des équations de Hamilton-Jacobi, Mathematics and Applications17. Springer-Verlag, Paris (1994).  
  5. R. Boucekkine, O. Licandro, L. Puch and F. del Rio, Vintage capital and the dynamics of the AK model. J. Econ. Theory120 (2005) 39–72.  
  6. H. Brezis, Analyse fonctionnelle, théorie et applications. Masson, Paris (1983).  
  7. G. Carlier and R. Tahraoui, On some optimal control problems governed by a state equation with memory. ESAIM: COCV14 (2008) 725–743.  
  8. M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. I. Uniqueness of viscosity solutions. J. Funct. Anal.62 (1985) 379–396.  
  9. M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. II. Existence of viscosity solutions. J. Funct. Anal.65 (1986) 368–405.  
  10. M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. III. J. Funct. Anal.68 (1986) 214–247.  
  11. M. Crandall and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions. IV. Hamiltonians with unbounded linear terms. J. Funct. Anal.90 (1990) 237–283.  
  12. M. Crandall and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions. V. Unbounded linear terms and B-continuous solutions. J. Funct. Anal.97 (1991) 417–465.  
  13. M. Crandall and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions. VI. Nonlinear A and Tataru's method refined, in Evolution equations, control theory, and biomathematics, Lect. Notes Pure Appl. Math.155, Dekker, New York (1994) 51–89.  
  14. I. Elsanosi, B. Øksendal and A. Sulem, Some solvable stochastic control problems with delay. Stochast. Stochast. Rep.71 (2000) 69–89.  
  15. G. Fabbri, Viscosity solutions to delay differential equations in demo-economy. Math. Popul. Stud.15 (2008) 27–54.  
  16. G. Fabbri, S. Faggian and F. Gozzi, On dynamic programming in economic models governed by DDEs. Math. Popul. Stud.15 (2008) 267–290.  
  17. S. Faggian and F. Gozzi, On the dynamic programming approach for optimal control problems of PDE's with age structure. Math. Popul. Stud.11 (2004) 233–270.  
  18. F. Gozzi and C. Marinelli, Stochastic optimal control of delay equations arising in advertising models, in Stochastic partial differential equations and applicationsVII, Chapman & Hall, Boca Raton, Lect. Notes Pure Appl. Math.245 (2006) 133–148.  
  19. V.B. Kolmanovskii and L.E. Shaikhet, Control of systems with aftereffect, Translations of Mathematical Monographs. American Mathematical Society, Providence, USA (1996).  
  20. B. Larssen and N.H. Risebro, When are HJB-equations in stochastic control of delay systems finite dimensional? Stochastic Anal. Appl.21 (2003) 643–671.  
  21. L. Samassi and R. Tahraoui, Comment établir des conditions nécessaires d'optimalité dans les problèmes de contrôle dont certains arguments sont déviés ? C. R. Math. Acad. Sci. Paris338 (2004) 611–616.  
  22. L. Samassi and R. Tahraoui, How to state necessary optimality conditions for control problems with deviating arguments? ESAIM: COCV14 (2008) 381–409.  

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.