Adaptive control of discrete time Markov processes by the large deviations method

T. Duncan; B. Pasik-Duncan; Łukasz Stettner

Applicationes Mathematicae (2000)

  • Volume: 27, Issue: 3, page 265-285
  • ISSN: 1233-7234

Abstract

top
Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.

How to cite

top

Duncan, T., Pasik-Duncan, B., and Stettner, Łukasz. "Adaptive control of discrete time Markov processes by the large deviations method." Applicationes Mathematicae 27.3 (2000): 265-285. <http://eudml.org/doc/219273>.

@article{Duncan2000,
abstract = {Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.},
author = {Duncan, T., Pasik-Duncan, B., Stettner, Łukasz},
journal = {Applicationes Mathematicae},
keywords = {discrete time controlled Markov processes; adaptive control; large deviations},
language = {eng},
number = {3},
pages = {265-285},
title = {Adaptive control of discrete time Markov processes by the large deviations method},
url = {http://eudml.org/doc/219273},
volume = {27},
year = {2000},
}

TY - JOUR
AU - Duncan, T.
AU - Pasik-Duncan, B.
AU - Stettner, Łukasz
TI - Adaptive control of discrete time Markov processes by the large deviations method
JO - Applicationes Mathematicae
PY - 2000
VL - 27
IS - 3
SP - 265
EP - 285
AB - Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.
LA - eng
KW - discrete time controlled Markov processes; adaptive control; large deviations
UR - http://eudml.org/doc/219273
ER -

References

top
  1. [1] G. B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics Stochastics Rep. 54 (1995), 301-316. Zbl0855.93103
  2. [2] M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, I, Comm. Pure Appl. Math. 28 (1975), 1-47. Zbl0323.60069
  3. [3] M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time--III, ibid. 29 (1976), 389-461. Zbl0348.60032
  4. [4] M. Duflo, Formule de Chernoff pour des chaînes de Markov ( d'après Donsker et Varadhan), in: Grandes déviations et applications statistiques, Séminaire Orsay 1977-78, Astérisque 68 (1979), 99-124. 
  5. [5] T. E. Duncan, B. Pasik-Duncan and Ł. Stettner, Discretized maximum likelihood and almost optimal control of ergodic Markov models, SIAM J. Control Optim. 36 (1998), 422-446. Zbl0914.93076
  6. [6] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, 1976. 
  7. [7] N. Maigret, Majorations de Chernoff pour des chaînes de Markov contrôlées, Z. Wahrsch. Verw. Gebiete 51 (1980), 133-151. Zbl0397.60051
  8. [8] N. Maigret, Statistiques des chaînes controlés Felleriennes, in: Grandes déviations et applications statistiques, Séminaire Orsay, 1977-1978, Astérisque 68 (1979), 143-169. 
  9. [9] Ł. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim. 27 (1993), 161-177. Zbl0769.93084

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.