Bayesian parameter estimation and adaptive control of Markov processes with time-averaged cost

V. Borkar; S. Associate

Displaying similar documents to “Bayesian parameter estimation and adaptive control of Markov processes with time-averaged cost”

Recursive self-tuning control of finite Markov chains

Vivek Borkar (1997)

Applicationes Mathematicae

Similarity:

A recursive self-tuning control scheme for finite Markov chains is proposed wherein the unknown parameter is estimated by a stochastic approximation scheme for maximizing the log-likelihood function and the control is obtained via a relative value iteration algorithm. The analysis uses the asymptotic o.d.e.s associated with these.

Estimation of hidden Markov models for a partially observed risk sensitive control problem

Bernard Frankpitt, John S. Baras (1998)

Kybernetika

Similarity:

This paper provides a summary of our recent work on the problem of combined estimation and control of systems described by finite state, hidden Markov models. We establish the stochastic framework for the problem, formulate a separated control policy with risk-sensitive cost functional, describe an estimation scheme for the parameters of the hidden Markov model that describes the plant, and finally indicate how the combined estimation and control problem can be re-formulated in a framework...

Ergodic control of partially observed Markov processes with equivalent transition probabilities

Łukasz Stettner (1993)

Applicationes Mathematicae

Similarity:

Optimal control with long run average cost functional of a partially observed Markov process is considered. Under the assumption that the transition probabilities are equivalent, the existence of the solution to the Bellman equation is shown, with the use of which optimal strategies are constructed.

Necessary and sufficient optimality conditions for average reward of controlled Markov chains

Karel Sladký (1973)

Kybernetika

Similarity:

Approximation and adaptive control of Markov processes: Average reward criterion

Onésimo Hernández-Lerma (1987)

Kybernetika

Similarity:

Adaptive control for discrete-time Markov processes with unbounded costs: Discounted criterion

Evgueni I. Gordienko, J. Adolfo Minjárez-Sosa (1998)

Kybernetika

Similarity:

We study the adaptive control problem for discrete-time Markov control processes with Borel state and action spaces and possibly unbounded one-stage costs. The processes are given by recurrent equations $x_{t + 1} = F (x_{t}, a_{t}, ξ_{t}), t = 0, 1, ...$ with i.i.d. $ℜ^{k}$ -valued random vectors $ξ_{t}$ whose density $ρ$ is unknown. Assuming observability of $ξ_{t}$ we propose the procedure of statistical estimation of $ρ$ that allows us to prove discounted asymptotic optimality of two types of adaptive policies used early for the processes with bounded...