A generalization of Ueno's inequality for n-step transition probabilities

Andrzej Nowak

Applicationes Mathematicae (1998)

  • Volume: 25, Issue: 3, page 295-299
  • ISSN: 1233-7234

Abstract

top
We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.

How to cite

top

Nowak, Andrzej. "A generalization of Ueno's inequality for n-step transition probabilities." Applicationes Mathematicae 25.3 (1998): 295-299. <http://eudml.org/doc/219204>.

@article{Nowak1998,
abstract = {We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.},
author = {Nowak, Andrzej},
journal = {Applicationes Mathematicae},
keywords = {adaptive control; transition probabilities; stochastic control; Markov chains},
language = {eng},
number = {3},
pages = {295-299},
title = {A generalization of Ueno's inequality for n-step transition probabilities},
url = {http://eudml.org/doc/219204},
volume = {25},
year = {1998},
}

TY - JOUR
AU - Nowak, Andrzej
TI - A generalization of Ueno's inequality for n-step transition probabilities
JO - Applicationes Mathematicae
PY - 1998
VL - 25
IS - 3
SP - 295
EP - 299
AB - We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.
LA - eng
KW - adaptive control; transition probabilities; stochastic control; Markov chains
UR - http://eudml.org/doc/219204
ER -

References

top
  1. [1] R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972. 
  2. [2] J. P. Georgin, Contrôle de chaînes de Markov sur des espaces arbitraires, Ann. Inst. H. Poincaré Sér. B 14 (1978), 255-277. Zbl0391.60066
  3. [3] O. Hernandez-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989. 
  4. [4] N. W. Kartashov, Criteria for uniform ergodicity and strong stability of Markov chains in general state space, Probab. Theory Math. Statist. 30 (1984), 65-81. 
  5. [5] G. B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics and Stochastics Reports 54 (1995), 301-316. Zbl0855.93103
  6. [6] A. S. Nowak and E. Altman, ε-Nash equilibria in stochastic games with uncountable state space and unbounded cost, Technical Report, Institute of Mathematics, Wrocław University of Technology, 1998. 
  7. [7] W. J. Runggaldier and Ł. Stettner, Approximations of Discrete Time Partially Observed Control Problems, Appl. Math. Monographs 6, C.N.R., Pisa, 1994. 
  8. [8] Ł. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim. 27 (1993), 161-177. Zbl0769.93084
  9. [9] T. Ueno, Some limit theorems for temporally discrete Markov processes, J. Fac. Sci. Univ. Tokyo 7 (1957), 449-462. Zbl0077.33201

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.