# Adaptive control of discrete time Markov processes by the large deviations method

T. Duncan; B. Pasik-Duncan; Łukasz Stettner

Applicationes Mathematicae (2000)

- Volume: 27, Issue: 3, page 265-285
- ISSN: 1233-7234

## Access Full Article

top## Abstract

top## How to cite

topDuncan, T., Pasik-Duncan, B., and Stettner, Łukasz. "Adaptive control of discrete time Markov processes by the large deviations method." Applicationes Mathematicae 27.3 (2000): 265-285. <http://eudml.org/doc/219273>.

@article{Duncan2000,

abstract = {Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.},

author = {Duncan, T., Pasik-Duncan, B., Stettner, Łukasz},

journal = {Applicationes Mathematicae},

keywords = {discrete time controlled Markov processes; adaptive control; large deviations},

language = {eng},

number = {3},

pages = {265-285},

title = {Adaptive control of discrete time Markov processes by the large deviations method},

url = {http://eudml.org/doc/219273},

volume = {27},

year = {2000},

}

TY - JOUR

AU - Duncan, T.

AU - Pasik-Duncan, B.

AU - Stettner, Łukasz

TI - Adaptive control of discrete time Markov processes by the large deviations method

JO - Applicationes Mathematicae

PY - 2000

VL - 27

IS - 3

SP - 265

EP - 285

AB - Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.

LA - eng

KW - discrete time controlled Markov processes; adaptive control; large deviations

UR - http://eudml.org/doc/219273

ER -

## References

top- [1] G. B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics Stochastics Rep. 54 (1995), 301-316. Zbl0855.93103
- [2] M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, I, Comm. Pure Appl. Math. 28 (1975), 1-47. Zbl0323.60069
- [3] M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time--III, ibid. 29 (1976), 389-461. Zbl0348.60032
- [4] M. Duflo, Formule de Chernoff pour des chaînes de Markov ( d'après Donsker et Varadhan), in: Grandes déviations et applications statistiques, Séminaire Orsay 1977-78, Astérisque 68 (1979), 99-124.
- [5] T. E. Duncan, B. Pasik-Duncan and Ł. Stettner, Discretized maximum likelihood and almost optimal control of ergodic Markov models, SIAM J. Control Optim. 36 (1998), 422-446. Zbl0914.93076
- [6] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, 1976.
- [7] N. Maigret, Majorations de Chernoff pour des chaînes de Markov contrôlées, Z. Wahrsch. Verw. Gebiete 51 (1980), 133-151. Zbl0397.60051
- [8] N. Maigret, Statistiques des chaînes controlés Felleriennes, in: Grandes déviations et applications statistiques, Séminaire Orsay, 1977-1978, Astérisque 68 (1979), 143-169.
- [9] Ł. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim. 27 (1993), 161-177. Zbl0769.93084

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.