A deterministic affine-quadratic optimal control problem

Yuanchang Wang; Jiongmin Yong

ESAIM: Control, Optimisation and Calculus of Variations (2014)

  • Volume: 20, Issue: 3, page 633-661
  • ISSN: 1292-8119

Abstract

top
A deterministic affine-quadratic optimal control problem is considered. Due to the nature of the problem, optimal controls exist under some very mild conditions. Further, it is shown that under some assumptions, the optimal control is unique which leads to the differentiability of the value function. Therefore, the value function satisfies the corresponding Hamilton–Jacobi–Bellman equation in the classical sense, and the optimal control admits a state feedback representation. Under some additional conditions, it is shown that the value function is actually twice differentiable and the so-called quasi-Riccati equation is derived, whose solution can be used to construct the state feedback representation for the optimal control.

How to cite

top

Wang, Yuanchang, and Yong, Jiongmin. "A deterministic affine-quadratic optimal control problem." ESAIM: Control, Optimisation and Calculus of Variations 20.3 (2014): 633-661. <http://eudml.org/doc/272934>.

@article{Wang2014,
abstract = {A deterministic affine-quadratic optimal control problem is considered. Due to the nature of the problem, optimal controls exist under some very mild conditions. Further, it is shown that under some assumptions, the optimal control is unique which leads to the differentiability of the value function. Therefore, the value function satisfies the corresponding Hamilton–Jacobi–Bellman equation in the classical sense, and the optimal control admits a state feedback representation. Under some additional conditions, it is shown that the value function is actually twice differentiable and the so-called quasi-Riccati equation is derived, whose solution can be used to construct the state feedback representation for the optimal control.},
author = {Wang, Yuanchang, Yong, Jiongmin},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
keywords = {affine quadratic optimal control; dynamic programming; Hamilton–Jacobi–Bellman equation; quasi-Riccati equation; state feedback representation; affine-quadratic optimal control problem; Hamilton-Jacobi-Bellman equation},
language = {eng},
number = {3},
pages = {633-661},
publisher = {EDP-Sciences},
title = {A deterministic affine-quadratic optimal control problem},
url = {http://eudml.org/doc/272934},
volume = {20},
year = {2014},
}

TY - JOUR
AU - Wang, Yuanchang
AU - Yong, Jiongmin
TI - A deterministic affine-quadratic optimal control problem
JO - ESAIM: Control, Optimisation and Calculus of Variations
PY - 2014
PB - EDP-Sciences
VL - 20
IS - 3
SP - 633
EP - 661
AB - A deterministic affine-quadratic optimal control problem is considered. Due to the nature of the problem, optimal controls exist under some very mild conditions. Further, it is shown that under some assumptions, the optimal control is unique which leads to the differentiability of the value function. Therefore, the value function satisfies the corresponding Hamilton–Jacobi–Bellman equation in the classical sense, and the optimal control admits a state feedback representation. Under some additional conditions, it is shown that the value function is actually twice differentiable and the so-called quasi-Riccati equation is derived, whose solution can be used to construct the state feedback representation for the optimal control.
LA - eng
KW - affine quadratic optimal control; dynamic programming; Hamilton–Jacobi–Bellman equation; quasi-Riccati equation; state feedback representation; affine-quadratic optimal control problem; Hamilton-Jacobi-Bellman equation
UR - http://eudml.org/doc/272934
ER -

References

top
  1. [1] S.P. Banks and T. Cimen, Global optimal feedback control for general nonlinear systems with nonquadratic performance criteria. System Control Lett.53 (2004) 327–346. Zbl1157.49313MR2097793
  2. [2] S.P. Banks and T. Cimen, Optimal control of nonlinear systems, Optimization and Control with Applications. In vol. 96 of Appl. Optim. Springer, New York (2005) 353–367. Zbl1089.49034MR2144384
  3. [3] H.T. Banks, B.M. Lewis and H.T. Tran, Nonlinear feedback controllers and compensators: a state-dependent Riccati equation approach. Comput. Optim. Appl.37 (2007) 177–218. Zbl1117.49032MR2325656
  4. [4] M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Birkhäuser, Boston (1997). Zbl0890.49011MR1484411
  5. [5] M. Bardi and F. DaLio, On the Bellman equation for some unbounded control problems. Nonlinear Differ. Eqs. Appl.4 (1997) 491–510. Zbl0894.49017MR1485734
  6. [6] L.M. Benveniste and J.A. Scheinkman, On the differentiability of the value function in dynamic models of economics. Econometrica47 (1979) 727–732. Zbl0435.90031MR533081
  7. [7] L.D. Berkovitz, Optimal Control Theory. Springer-Verlag, New York (1974). Zbl0295.49001MR372707
  8. [8] J.F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems. Springer, New York (2000). Zbl0966.49001MR1756264
  9. [9] P. Cannarsa and H. Frankowska, Some characterizatins of optimal trajecotries in control theory. SIAM J. Control Optim.29 (1991) 1322–1347. Zbl0744.49011MR1132185
  10. [10] T. Cimen, State-dependent Riccati equation (SDRE) control: a survey. Proc. 17th World Congress IFAC (2008) 3761–3775. 
  11. [11] H. Frankowska, Value Function in Optimal Control, Mathematical Control Theory, Part 1, 2 (2001) 516–653. Zbl1098.49501MR1972793
  12. [12] T. Hildebrandt and L. Graves, Implicit functions and their differentials in general analysis. Trans. Amer. Math. Soc.29 (1927) 127–153. Zbl53.0234.02MR1501380JFM53.0234.02
  13. [13] Y. Hu and S. Peng, Solution of forward-backward stochastic differential equations. Probab. Theory Rel. Fields103 (1995) 273–283. Zbl0831.60065MR1355060
  14. [14] R.E. Kalman, Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana5 (1960) 102–119. Zbl0112.06303MR127472
  15. [15] J. Ma and J. Yong, Forward-Backward Stochastic Differential Equations and Their Applications. Vol. 1702 of Lect. Notes Math. Springer-Verlag (1999). Zbl0927.60004MR1704232
  16. [16] H. Qiu and J. Yong, Hamilton-Jacobi equations and two-person zero-sum differential games with unbounded controls. ESAIM: COCV 19 (2013) 404–437. Zbl1263.49024MR3049717
  17. [17] J.P. Rincón-Zapatero and M.S. Santos, Differentiability of the value function in continuous-time economic models. J. Math. Anal. Appl.394 (2012) 305–323. Zbl1251.91046MR2926223
  18. [18] J. Yong, Finding adapted solutions of forward-backward stochastic differential equations – method of continuation, Probab. Theory Rel. Fields107 (1997) 537–572. Zbl0883.60053MR1440146
  19. [19] J. Yong, Stochastic optimal control and forward-backward stochastic differential equations. Comput. Appl. Math.21 (2002) 369–403. Zbl1123.60313MR2009959
  20. [20] J. Yong, Forward backward stochastic differential equations with mixed initial and terminal conditions. Trans. AMS362 (2010) 1047–1096. Zbl1185.60067MR2551515
  21. [21] J. Yong and X.Y. Zhou, Stochastic Control: Hamiltonian Systems and HJB Equations. Springer-Verlag (1999). Zbl0943.93002MR1696772
  22. [22] Y. You, A nonquadratic Bolza problem and a quasi-Riccati equation for distributed parameter systems. SIAM J. Control Optim.25 (1987) 905–920. Zbl0632.49004MR893989
  23. [23] Y. You, Synthesis of time-variant optimal control with nonquadratic criteria. J. Math. Anal. Appl.209 (1997) 662–682. Zbl0872.49015MR1474631
  24. [24] E. Zeidler, Nonlinear Functional Analysis and Its Applications, I: Fixed-Point Theorems. Springer-Verlag, New York (1986) 150–151. Zbl0583.47050MR816732
  25. [25] E. Zeidler, Nonlinear Functional Analysis and Its Applications, II/B: Nonlinear Monotone Operators. Springer-Verlag, New York (1990). Zbl0684.47029MR1033498

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.