Mean-Field Optimal Control

Massimo Fornasier; Francesco Solombrino

ESAIM: Control, Optimisation and Calculus of Variations (2014)

  • Volume: 20, Issue: 4, page 1123-1152
  • ISSN: 1292-8119

Abstract

top
We introduce the concept of mean-field optimal control which is the rigorous limit process connecting finite dimensional optimal control problems with ODE constraints modeling multi-agent interactions to an infinite dimensional optimal control problem with a constraint given by a PDE of Vlasov-type, governing the dynamics of the probability distribution of interacting agents. While in the classical mean-field theory one studies the behavior of a large number of small individuals freely interacting with each other, by simplifying the effect of all the other individuals on any given individual by a single averaged effect, we address the situation where the individuals are actually influenced also by an external policy maker, and we propagate its effect for the number N of individuals going to infinity. On the one hand, from a modeling point of view, we take into account also that the policy maker is constrained to act according to optimal strategies promoting its most parsimonious interaction with the group of individuals. This will be realized by considering cost functionals including L1-norm terms penalizing a broadly distributed control of the group, while promoting its sparsity. On the other hand, from the analysis point of view, and for the sake of generality, we consider broader classes of convex control penalizations. In order to develop this new concept of limit rigorously, we need to carefully combine the classical concept of mean-field limit, connecting the finite dimensional system of ODE describing the dynamics of each individual of the group to the PDE describing the dynamics of the respective probability distribution, with the well-known concept of Γ-convergence to show that optimal strategies for the finite dimensional problems converge to optimal strategies of the infinite dimensional problem.

How to cite

top

Fornasier, Massimo, and Solombrino, Francesco. "Mean-Field Optimal Control." ESAIM: Control, Optimisation and Calculus of Variations 20.4 (2014): 1123-1152. <http://eudml.org/doc/272875>.

@article{Fornasier2014,
abstract = {We introduce the concept of mean-field optimal control which is the rigorous limit process connecting finite dimensional optimal control problems with ODE constraints modeling multi-agent interactions to an infinite dimensional optimal control problem with a constraint given by a PDE of Vlasov-type, governing the dynamics of the probability distribution of interacting agents. While in the classical mean-field theory one studies the behavior of a large number of small individuals freely interacting with each other, by simplifying the effect of all the other individuals on any given individual by a single averaged effect, we address the situation where the individuals are actually influenced also by an external policy maker, and we propagate its effect for the number N of individuals going to infinity. On the one hand, from a modeling point of view, we take into account also that the policy maker is constrained to act according to optimal strategies promoting its most parsimonious interaction with the group of individuals. This will be realized by considering cost functionals including L1-norm terms penalizing a broadly distributed control of the group, while promoting its sparsity. On the other hand, from the analysis point of view, and for the sake of generality, we consider broader classes of convex control penalizations. In order to develop this new concept of limit rigorously, we need to carefully combine the classical concept of mean-field limit, connecting the finite dimensional system of ODE describing the dynamics of each individual of the group to the PDE describing the dynamics of the respective probability distribution, with the well-known concept of Γ-convergence to show that optimal strategies for the finite dimensional problems converge to optimal strategies of the infinite dimensional problem.},
author = {Fornasier, Massimo, Solombrino, Francesco},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
keywords = {Sparse optimal control; mean-field limit; Γ-limit; optimal control with ODE constraints; optimal control with PDE constraints; sparse optimal control; -limit; ODE constraints; PDE constraints; cost functional},
language = {eng},
number = {4},
pages = {1123-1152},
publisher = {EDP-Sciences},
title = {Mean-Field Optimal Control},
url = {http://eudml.org/doc/272875},
volume = {20},
year = {2014},
}

TY - JOUR
AU - Fornasier, Massimo
AU - Solombrino, Francesco
TI - Mean-Field Optimal Control
JO - ESAIM: Control, Optimisation and Calculus of Variations
PY - 2014
PB - EDP-Sciences
VL - 20
IS - 4
SP - 1123
EP - 1152
AB - We introduce the concept of mean-field optimal control which is the rigorous limit process connecting finite dimensional optimal control problems with ODE constraints modeling multi-agent interactions to an infinite dimensional optimal control problem with a constraint given by a PDE of Vlasov-type, governing the dynamics of the probability distribution of interacting agents. While in the classical mean-field theory one studies the behavior of a large number of small individuals freely interacting with each other, by simplifying the effect of all the other individuals on any given individual by a single averaged effect, we address the situation where the individuals are actually influenced also by an external policy maker, and we propagate its effect for the number N of individuals going to infinity. On the one hand, from a modeling point of view, we take into account also that the policy maker is constrained to act according to optimal strategies promoting its most parsimonious interaction with the group of individuals. This will be realized by considering cost functionals including L1-norm terms penalizing a broadly distributed control of the group, while promoting its sparsity. On the other hand, from the analysis point of view, and for the sake of generality, we consider broader classes of convex control penalizations. In order to develop this new concept of limit rigorously, we need to carefully combine the classical concept of mean-field limit, connecting the finite dimensional system of ODE describing the dynamics of each individual of the group to the PDE describing the dynamics of the respective probability distribution, with the well-known concept of Γ-convergence to show that optimal strategies for the finite dimensional problems converge to optimal strategies of the infinite dimensional problem.
LA - eng
KW - Sparse optimal control; mean-field limit; Γ-limit; optimal control with ODE constraints; optimal control with PDE constraints; sparse optimal control; -limit; ODE constraints; PDE constraints; cost functional
UR - http://eudml.org/doc/272875
ER -

References

top
  1. [1] N. Ahmed and X. Ding, Controlled McKean-Vlasov equations. Commun. Appl. Anal.5 (2001) 183–206. Zbl1084.49506MR1844190
  2. [2] L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems. Oxford, Clarendon Press (2000). Zbl0957.49001MR1857292
  3. [3] L. Ambrosio, N. Gigli and G. Savaré, Gradient Flows in Metric Spaces and in the Space of Probability Measures. Lect. Math. ETH Zürich 2nd, edition. Birkhäuser Verlag, Basel (2008). Zbl1145.35001MR2401600
  4. [4] D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type. Appl. Math. Opt.63 (2011) 341–356. Zbl1215.49034MR2784835
  5. [5] M. Ballerini, N. Cabibbo, R. Candelier, A. Cavagna, E. Cisbani, L. Giardina, L. Lecomte, A. Orlandi, G. Parisi, A. Procaccini, M. Viale and V. Zdravkovic, Interaction ruling animal collective behavior depends on topological rather than metric distance: evidence from a field study. Proc. National Academy of Sci.105 (2008) 1232–1237. 
  6. [6] A. Bensoussan, J. Frehse and P. Yam, Mean field games and mean field type control theory. Springer, New York (2013). Zbl1287.93002MR3134900
  7. [7] A. Bressan and B. Piccoli, Introduction to the Mathematical Theory of Control, vol. 2 of AIMS Ser. Appl. Math.. American Institute of Mathematical Sciences (AIMS), Springfield, MO (2007). Zbl1127.93002MR2347697
  8. [8] R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for sdes of mean-field type. Appl. Math. Opt.64 (2011) 197–216. Zbl1245.49036MR2822408
  9. [9] S. Camazine, J. Deneubourg, N. Franks, J. Sneyd, G. Theraulaz and E. Bonabeau, Self-Organization in Biological Systems. Princeton University Press (2003). Zbl1130.92009MR2343706
  10. [10] J.A. Cañizo, J.A. Carrillo and J. Rosado, A well-posedness theory in measures for some kinetic models of collective motion. Math. Model. Meth. Appl. Sci.21 (2011) 515–539. Zbl1218.35005
  11. [11] M. Caponigro, M. Fornasier, B. Piccoli and E. Trélat, Sparse stabilization and control of the Cucker−Smale model. Preprint: arXiv:1210.5739 (2012). Zbl1275.49003
  12. [12] J.A. Carrillo, Y.-P. Choi and M. Hauray, The derivation of swarming models: mean-field limit and Wasserstein distances. Preprint: arXiv:1304.5776 (2013). 
  13. [13] J.A. Carrillo, M.R. D’Orsogna and V. Panferov, Double milling in self-propelled swarms from kinetic theory. Kinet. Relat. Models2 (2009) 363–378. Zbl1195.92069MR2507454
  14. [14] J. A. Carrillo, M. Fornasier, G. Toscani and F. Vecil, Particle, kinetic, and hydrodynamic models of swarming, in Math. Modeling of Collective Behavior in Socio-Economic and Life Sci., edited by G. Naldi, L. Pareschi, G. Toscani and N. Bellomo. Model. Simul. Sci. Engrg. Technol. Birkhäuser, Boston (2010) 297–336. Zbl1211.91213MR2744704
  15. [15] E. Casas, C. Clason and K. Kunisch, Approximation of elliptic control problems in measure spaces with sparse solutions. SIAM J. Control Optim.50 (2012) 1735–1752. Zbl06111073MR2974716
  16. [16] Y. Chuang, M. D’Orsogna, D. Marthaler, A. Bertozzi and L. Chayes, State transition and the continuum limit for the 2D interacting, self-propelled particle system. Physica D232 (2007) 33–47. Zbl05182405MR2369988
  17. [17] Y. Chuang, Y. Huang, M. D’Orsogna and A. Bertozzi, Multi-vehicle flocking: scalability of cooperative control algorithms using pairwise potentials. IEEE Int. Conference on Robotics and Automation (2007) 2292–2299. 
  18. [18] C. Clason and K. Kunisch, A duality-based approach to elliptic control problems in non-reflexive Banach spaces. ESAIM: COCV 17 (2011) 243–266. Zbl1213.49041MR2775195
  19. [19] C. Clason and K. Kunisch, A measure space approach to optimal source placement. Comput. Optim. Appl.53 (2012) 155–171. Zbl1258.49041MR2964839
  20. [20] I. Couzin and N. Franks, Self-organized lane formation and optimized traffic flow in army ants. Proc. R. Soc. London B270 (2002) 139–146. 
  21. [21] I. Couzin, J. Krause, N. Franks and S. Levin, Effective leadership and decision making in animal groups on the move. Nature433 (2005) 513–516. 
  22. [22] A. J. Craig and I. Flügge-Lotz, Investigation of optimal control with a minimum-fuel consumption criterion for a fourth-order plant with two control inputs; synthesis of an efficient suboptimal control. J. Basic Engrg.87 (1965) 39–58. 
  23. [23] E. Cristiani, B. Piccoli and A. Tosin, Modeling self-organization in pedestrians and animal groups from macroscopic and microscopic viewpoints, in Mathematical Modeling of Collective Behavior in Socio-Economic and Life Sciences, edited by G. Naldi, L. Pareschi, G. Toscani and N. Bellomo. Model. Simul. Sci. Engrg. Technol. Birkhäuser, Boston (2010). Zbl1211.91214MR2744705
  24. [24] E. Cristiani, B. Piccoli and A. Tosin, Multiscale modeling of granular flows with application to crowd dynamics. Multiscale Model. Simul.9 (2011) 155–182. Zbl1221.35232MR2769993
  25. [25] F. Cucker and J.-G. Dong, A general collision-avoiding flocking framework. IEEE Trans. Automat. Control56 (2011) 1124–1129. MR2815917
  26. [26] F. Cucker and E. Mordecki, Flocking in noisy environments. J. Math. Pures Appl.89 (2008) 278–296. Zbl1273.91404MR2401690
  27. [27] F. Cucker and S. Smale, Emergent behavior in flocks. IEEE Trans. Automat. Control 52 (2007) 852–862,. MR2324245
  28. [28] F. Cucker and S. Smale, On the mathematics of emergence. Japan J. Math.2 (2007) 197–227. Zbl1166.92323MR2295620
  29. [29] F. Cucker, S. Smale and D. Zhou, Modeling language evolution. Found. Comput. Math.4 (2004) 315–343. Zbl1083.68131MR2078666
  30. [30] G. Dal Maso, An Introduction to Γ-Convergence. Progress in Nonlinear Differ. Eqs. Appl., vol. 8. Birkhäuser Boston Inc., Boston, MA (1993). Zbl0816.49001MR1201152
  31. [31] H. Federer, Geometric Measure Theory. Die Grundlehren der mathematischen Wissenschaften in Einzeldarstellungen, vol. 153. Springer-Verlag, Berlin, Heidelberg, New York (1969). Zbl0176.00801MR257325
  32. [32] A.F. Filippov, Differential equations with Discontinuous Righthand Sides. Vol. 18 of Math. Appl. (Soviet Series). Translated from the Russian. Kluwer Academic Publishers Group, Dordrecht (1988). Zbl0664.34001MR1028776
  33. [33] R. Glowinski, Numerical Methods for Nonlinear Variational Problems. Scientific Comput. Springer-Verlag, Berlin (2008). Reprint of the 1984 original. Zbl1139.65050MR2423313
  34. [34] G. Grégoire and H. Chaté, Onset of collective and cohesive motion. Phys. Rev. Lett. 92 (2004). 
  35. [35] R. Herzog, G. Stadler and G. Wachsmuth, Directional sparsity in optimal control of partial differential equations. SIAM J. Control Optim.50 (2012) 943–963. Zbl1244.49038MR2914236
  36. [36] M. Huang, P. Caines and R. Malhamé, Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions. Proc. of the 42nd IEEE Conference on Decision and Control Maui, Hawaii USA (2003) 98–103. 
  37. [37] A. Jadbabaie, J. Lin and A.S. Morse, Correction to: “Coordination of groups of mobile autonomous agents using nearest neighbor rules” [48 (2003) 988–1001; MR 1986266]. IEEE Trans. Automat. Control 48 (2003) 1675. MR1986266
  38. [38] J. Ke, J. Minett, C.-P. Au and W.-Y. Wang, Self-organization and selection in the emergence of vocabulary. Complexity7 (2002) 41–54. MR1911875
  39. [39] E. F. Keller and L.A. Segel, Initiation of slime mold aggregation viewed as an instability. J. Theoret. Biol.26 (1970) 399–415. Zbl1170.92306
  40. [40] A. Koch and D. White, The social lifestyle of myxobacteria. Bioessays20 (1998) 1030–1038. 
  41. [41] J.-M. Lasry and P.-L. Lions, Mean field games. Japan J. Math.2 (2007) 229–260. Zbl1156.91321MR2295621
  42. [42] N. Leonard and E. Fiorelli, Virtual leaders, artificial potentials and coordinated control of groups. Proc. of 40th IEEE Conf. Decision Contr. (2001) 2968–2973. 
  43. [43] H. Niwa, Self-organizing dynamic model of fish schooling. J. Theoret. Biol.171 (1994) 123–136. 
  44. [44] M. Nuorian, P. Caines and R. Malhamé, Synthesis of Cucker−Smale type flocking via mean field stochastic control theory: Nash equilibria. Proc. of 48th Allerton Conf. Comm., Cont. Comp., Monticello, Illinois (2010) 814–815. 
  45. [45] M. Nuorian, P. Caines and R. Malhamé, Mean field analysis of controlled Cucker−Smale type flocking: Linear analysis and perturbation equations. Proc. of 18th IFAC World Congress Milano, Italy (2011) 4471–4476. 
  46. [46] J. Parrish and L. Edelstein-Keshet, Complexity, pattern and evolutionary trade-offs in animal aggregation. Science294 (1999) 99–101. 
  47. [47] J. Parrish, S. Viscido and D. Gruenbaum, Self-organized fish schools: An examination of emergent properties. Biol. Bull.202 (2002) 296–305. 
  48. [48] L. Perea, G. Gómez and P. Elosegui, Extension of the Cucker–Smale control law to space flight formations. AIAA J. Guidance, Control, and Dynamics 32 2009527–537. 
  49. [49] B. Perthame, Mathematical tools for kinetic equations. Bull. Am. Math. Soc., New Ser. 41 (2004) 205–244. Zbl1151.82351MR2043752
  50. [50] B. Perthame, Transport Equations in Biology. Basel, Birkhäuser (2007). Zbl1185.92006MR2270822
  51. [51] K. Pieper and B. Vexler, A priori error analysis for discretization of sparse elliptic optimal control problems in measure space. SIAM J. Control Optim.51 (2013) 2788–2808. Zbl06227446MR3072225
  52. [52] Y. Privat, E. Trélat and E. Zuazua, Complexity and regularity of maximal energy domains for the wave equation with fixed initial data. Discrete Contin. Dyn. Syst. Ser. A. Zbl1332.93069
  53. [53] R. Rannacher and B. Vexler, Adaptive finite element discretization in PDE-based optimization. GAMM-Mitt. 33 (2010) 177–193. Zbl1207.49023MR2843958
  54. [54] W. Romey, Individual differences make a difference in the trajectories of simulated schools of fish. Ecol. Model.92 (1996) 65–77. 
  55. [55] M.B. Short, M. R. D’Orsogna, V.B. Pasour, G.E. Tita, P.J. Brantingham, A.L. Bertozzi and L.B. Chayes, A statistical model of criminal behavior. Math. Models Methods Appl. Sci.18 (2008) 1249–1267. Zbl1180.35530MR2438215
  56. [56] G. Stadler, Elliptic optimal control problems with L1-control cost and applications for the placement of control devices. Comput. Optim. Appl.44 (2009) 159–181. Zbl1185.49031MR2556849
  57. [57] K. Sugawara and M. Sano, Cooperative acceleration of task performance: Foraging behavior of interacting multi-robots system. Phys. D100 (1997) 343–354. Zbl0908.92036
  58. [58] J. Toner and Y. Tu, Long-range order in a two-dimensional dynamical xy model: How birds fly together. Phys. Rev. Lett.75 (1995) 4326–4329. 
  59. [59] T. Vicsek, A. Czirok, E. Ben-Jacob, I. Cohen and O. Shochet, Novel type of phase transition in a system of self-driven particles. Phys. Rev. Lett.75 (1995) 1226–1229. 
  60. [60] T. Vicsek and A. Zafeiris, Collective motion. Phys. Rep.517 (2012) 71–140. 
  61. [61] C. Villani, Optimal Transport, vol. 338. Grundlehren der Math. Wissenschaften, [Fundamental Principles of Mathematical Science]. Springer-Verlag, Berlin (2009). Old and new. Zbl1156.53003MR2459454
  62. [62] G. Vossen and H. Maurer, L1 minimization in optimal control and applications to robotics. Optim. Control Appl. Methods27 (2006) 301–321. MR2283487
  63. [63] G. Wachsmuth and D. Wachsmuth, Convergence and regularization results for optimal control problems with sparsity functional. ESAIM: COCV 17 (2011) 858–886. Zbl1228.49032MR2826983
  64. [64] C. Yates, R. Erban, C. Escudero, L. Couzin, J. Buhl, L. Kevrekidis, P. Maini and D. Sumpter, Inherent noise can facilitate coherence in collective swarm motion. Proc. Natl. Acad. Sci.106 (2009) 5464–5469. 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.