Viscosity solutions of the Bellman equation for exit time optimal control problems with non-Lipschitz dynamics

Michael Malisoff

ESAIM: Control, Optimisation and Calculus of Variations (2001)

  • Volume: 6, page 415-441
  • ISSN: 1292-8119

Abstract

top
We study the Bellman equation for undiscounted exit time optimal control problems with fully nonlinear lagrangians and fully nonlinear dynamics using the dynamic programming approach. We allow problems whose non-Lipschitz dynamics admit more than one solution trajectory for some choices of open loop controls and initial positions. We prove a uniqueness theorem which characterizes the value functions of these problems as the unique viscosity solutions of the corresponding Bellman equations that satisfy appropriate boundary conditions. We deduce that the value function for Sussmann’s Reflected Brachystochrone Problem for an arbitrary singleton target is the unique viscosity solution of the corresponding Bellman equation in the class of functions which are continuous in the plane, null at the target, and bounded below. Our results also apply to degenerate eikonal equations, and to problems whose targets can be unbounded and whose lagrangians vanish for some points in the state space which are outside the target, including Fuller’s Example.

How to cite

top

Malisoff, Michael. "Viscosity solutions of the Bellman equation for exit time optimal control problems with non-Lipschitz dynamics." ESAIM: Control, Optimisation and Calculus of Variations 6 (2001): 415-441. <http://eudml.org/doc/90601>.

@article{Malisoff2001,
abstract = {We study the Bellman equation for undiscounted exit time optimal control problems with fully nonlinear lagrangians and fully nonlinear dynamics using the dynamic programming approach. We allow problems whose non-Lipschitz dynamics admit more than one solution trajectory for some choices of open loop controls and initial positions. We prove a uniqueness theorem which characterizes the value functions of these problems as the unique viscosity solutions of the corresponding Bellman equations that satisfy appropriate boundary conditions. We deduce that the value function for Sussmann’s Reflected Brachystochrone Problem for an arbitrary singleton target is the unique viscosity solution of the corresponding Bellman equation in the class of functions which are continuous in the plane, null at the target, and bounded below. Our results also apply to degenerate eikonal equations, and to problems whose targets can be unbounded and whose lagrangians vanish for some points in the state space which are outside the target, including Fuller’s Example.},
author = {Malisoff, Michael},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
keywords = {viscosity solutions; dynamical systems; reflected brachystochrone problem; Bellman equation; exit time optimal control; dynamic programming; eikonal equations},
language = {eng},
pages = {415-441},
publisher = {EDP-Sciences},
title = {Viscosity solutions of the Bellman equation for exit time optimal control problems with non-Lipschitz dynamics},
url = {http://eudml.org/doc/90601},
volume = {6},
year = {2001},
}

TY - JOUR
AU - Malisoff, Michael
TI - Viscosity solutions of the Bellman equation for exit time optimal control problems with non-Lipschitz dynamics
JO - ESAIM: Control, Optimisation and Calculus of Variations
PY - 2001
PB - EDP-Sciences
VL - 6
SP - 415
EP - 441
AB - We study the Bellman equation for undiscounted exit time optimal control problems with fully nonlinear lagrangians and fully nonlinear dynamics using the dynamic programming approach. We allow problems whose non-Lipschitz dynamics admit more than one solution trajectory for some choices of open loop controls and initial positions. We prove a uniqueness theorem which characterizes the value functions of these problems as the unique viscosity solutions of the corresponding Bellman equations that satisfy appropriate boundary conditions. We deduce that the value function for Sussmann’s Reflected Brachystochrone Problem for an arbitrary singleton target is the unique viscosity solution of the corresponding Bellman equation in the class of functions which are continuous in the plane, null at the target, and bounded below. Our results also apply to degenerate eikonal equations, and to problems whose targets can be unbounded and whose lagrangians vanish for some points in the state space which are outside the target, including Fuller’s Example.
LA - eng
KW - viscosity solutions; dynamical systems; reflected brachystochrone problem; Bellman equation; exit time optimal control; dynamic programming; eikonal equations
UR - http://eudml.org/doc/90601
ER -

References

top
  1. [1] O. Alvarez, Bounded-from-below solutions of Hamilton–Jacobi equations. Differential Integral Equations 10 (1997) 419-436. Zbl0890.35026
  2. [2] M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Birkhäuser, Boston (1997). Zbl0890.49011
  3. [3] M. Bardi and F. Da Lio, On the Bellman equation for some unbounded control problems. NODEA Nonlinear Differential Equations Appl. 4 (1997) 276-285. Zbl0894.49017MR1485734
  4. [4] M. Bardi, M. Falcone and P. Soravia, Numerical methods for pursuit-evasion games and viscosity solutions, in Stochastic and Differential Games: Theory and Numerical Methods, edited by M. Bardi, T.E.S. Raghavan and T. Parthasarathy. Birkhäuser, Boston (1999). Zbl1018.49027MR1678284
  5. [5] M. Bardi and P. Soravia, Hamilton–Jacobi equations with singular boundary conditions on a free boundary and applications to differential games. Trans. Amer. Math. Soc. 325 (1991) 205-229. Zbl0732.35013
  6. [6] C. Castaing, Sur les multi-applications mesurables. RAIRO Oper. Res. 1 (1967). Zbl0153.08501MR223527
  7. [7] M.G. Crandall, H. Ishii and P.-L. Lions, User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. 27 (1992) 1-67. Zbl0755.35015
  8. [8] F. Da Lio, On the Bellman equation for infinite horizon problems with unbounded cost functional. Appl. Math. Optim. 41 (1999) 171-197. Zbl0952.49023MR1731417
  9. [9] W.H. Fleming and H.M. Soner, Controlled Markov Processes and Viscosity Solutions. Springer, New York (1993). Zbl0773.60070MR1199811
  10. [10] G.B. Folland, Real Analysis: Modern Techniques and their Applications. J. Wiley and Sons, New York (1984). Zbl0549.28001MR767633
  11. [11] H. Ishii, On representation of solutions of Hamilton–Jacobi equations with convex Hamiltonians, in Recent Topics in Nonlinear PDE II, edited by K. Masuda and M. Mimura. Kinokuniya Company, Tokyo (1985). 
  12. [12] V. Jurdjevic, Geometric Control Theory. Cambridge University Press (1997). Zbl0940.93005MR1425878
  13. [13] M. Malisoff, A remark on the Bellman equation for optimal control problems with exit times and noncoercing dynamics, in Proc. 38th IEEE Conf. on Decision and Control. Phoenix, AZ (1999) 877-881. 
  14. [14] M. Malisoff, Viscosity solutions of the Bellman equation for exit time optimal control problems with vanishing Lagrangians (submitted). Zbl1012.49024
  15. [15] P. Soravia, Pursuit-evasion problems and viscosity solutions of Isaacs equations. SIAM J. Control. Optim. 31 (1993) 604-623. Zbl0786.35018MR1214756
  16. [16] P. Soravia, Discontinuous viscosity solutions to Dirichlet problems for Hamilton–Jacobi equations with convex Hamiltonians. Comm. Partial Differential Equations 18 (1993) 1493-1514. Zbl0788.49028
  17. [17] P. Soravia, Optimality principles and representation formulas for viscosity solutions of Hamilton–Jacobi equations I: Equations of unbounded and degenerate control problems without uniqueness. Adv. Differential Equations 4 (1999) 275-296. Zbl0955.49016
  18. [18] P. Soravia, Optimal control with discontinuous running cost: Eikonal equation and shape from shading, in Proc. 39th IEEE CDC (to appear). 
  19. [19] P. Souganidis, Two-player, zero-sum differential games and viscosity solutions, in Stochastic and Differential Games: Theory and Numerical Methods, edited by M. Bardi, T.E.S. Raghavan and T. Parthasarathy. Birkhäuser, Boston (1999). Zbl0947.91014MR1678285
  20. [20] H.J. Sussmann, A general theorem on local controllability. SIAM J. Control Optim. 25 (1987) 158-194. Zbl0629.93012MR872457
  21. [21] H. Sussmann, From the Brachystochrone problem to the maximum principle, in Proc. of the 35th IEEE Conference on Decision and Control. IEEE Publications, New York (1996) 1588-1594. 
  22. [22] H.J. Sussmann, Geometry and optimal control, in Mathematical Control Theory, edited by J. Baillieul and J.C. Willems. Springer-Verlag, New York (1998) 140-198. Zbl1067.49500MR1661472
  23. [23] H.J. Sussmann and B. Piccoli, Regular synthesis and sufficient conditions for optimality. SISSA Preprint 68/96/M. SIAM J. Control Optim. (to appear). Zbl0961.93014MR1788064
  24. [24] J. Warga, Optimal Control of Differential and Functional Equations. Academic Press, New York (1972). Zbl0253.49001MR372708
  25. [25] M.I. Zelikin and V.F. Borisov, Theory of Chattering Control. Birkhäuser, Boston (1994). Zbl0820.70003MR1279383

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.