Time-discretization for controlled Markov processes. I. General approximation results

 Access to full text

 Full (PDF)

1. A. Bensoussan M. Robin, On the convergence of the discrete time dynamic programming equation for general semi-groups, SIAM J. Control Optim. 20 (1982), 1, 722-746. (1982) MR0667651
2. N. Christopeit, Discrete approximation of continuous time stochastic control systems, SIAM J. Control Optim. 21 (1983), 1, 17-40. (1983) Zbl0508.93065MR0688438
3. B. T. Doshi, Optimal control of the service rate in an $M\left|G\right|1$-queueing system, Adv. in Appl. Probab. 10 (1978), 682-701. (1978) Zbl0381.60086MR0499221
4. W. H. Fleming R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer Verlag, Berlin 1975. (1975) MR0454768
5. I. I. Gihman A. V. Skorohod, Controlled Stochastic Processes, Springer Verlag, Berlin 1979. (1979) MR0544839
6. U. G. Haussmann, A discrete approximation to optimal stochastic, In: Analysis and Optimization of Stochastic Systems, Academic Press, London 1980, pp. 229-241. (1980) Zbl0476.93082MR0592987
7. A. Hordijk F. A. Van der Duyn Schouten, Average optimal policies in Markov decision drift processes with applications to queueing and replacement model, Adv. in Appl. Probab. 15 (1983), 274-303. (1983) MR0698820
8. A. Hordijk F. A. Van der Duyn Schouten, Discretization and weak convergence in Markov decision drift processes, Math. Oper. Res. 9 (1984), 1, 112-141. (1984) MR0736642
9. A. Hordijk F. A. Van der Duyn Schouten, Markov decision drift processes; Conditions for optimality obtained by discretization, Math. Oper. Res. 10 (1985), 160-173. (1985) MR0787013
10. A. Hordijk F. A. Van der Duyn Schouten, On the optimality of $(s,S)$-policies in continuous review inventory models, SIAM J. Appl. Math. 46 (1986), 912-929. (1986) MR0859000
11. G. M. Koole, Stochastic Scheduling and Dynamic Programming, Ph.D. Thesis, University of Leiden 1992. (1992) 
12. T. G. Kurtz, Extensions of Trotter's operator semigroup approximations theorems, J. Funct. Anal. 3 (1969), 111-132. (1969) MR0242016
13. H. J. Kushner, Probability Methods for Approximation in Stochastic Control and for Elliptic Equations, Academic Press, New York 1977. (1977) MR0469468
14. P. D. Lax R. D. Richtmeyer, Survey of the stability of linear finite difference equations, Comm. Pure Appl. Math. 9 (1956), 267-293. (1956) MR0079204
15. T. Meis U. Marcowitz, Numerical Solution of Partial Differential Equations, Springer Verlag, Berlin 1981. (1981) MR0617910
16. H. J. Plum, Impulsive and continuously acting control of jump processes -- Time discretization, Stochastics and Stochastic Reports 36 (1991), 163-192. (1991) Zbl0739.60076MR1128492
17. R. Rishel, Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control, SIAM J. Control 8 (1970), 4, 559-571. (1970) Zbl0206.45804MR0274161
18. R. Rishel, Controls optimal from the toward and dynamic programming for systems of controlled jump processes, Math. Programming Study 6 (1976), 125-153. (1976) MR0479611
19. F. A. Van der Duyn Schouten, Markov Decision Processes with Continuous Time Parameter, Mathematical Centre Tract 164, Amsterdam 1983. (1983) Zbl0519.90052MR0709095
20. N. M. Van Dijk, Controlled Markov Processes; Time Discretization/Networks of Queues, Ph.D. Thesis, University of Leiden 1983. (1983) 
21. N. M. Van Dijk, On the finite horizon Bellman equation for controlled Markov jump models with unbounded characteristics: existence and approximations, Stochastic Process. Appl. 28 (1988), 141-157. (1988) MR0936380