Automatic stochastic control of impulses on a three-dimensional crystallographic lattice
Average cost Markov control processes with weighted norms: existence of canonical policies
This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.
Average cost Markov control processes with weighted norms: value iteration
This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.
Bang-bang controls in the singular perturbations limit
Bayesian estimation of mixtures with dynamic transitions and known component parameters
Probabilistic mixtures provide flexible “universal” approximation of probability density functions. Their wide use is enabled by the availability of a range of efficient estimation algorithms. Among them, quasi-Bayesian estimation plays a prominent role as it runs “naturally” in one-pass mode. This is important in on-line applications and/or extensive databases. It even copes with dynamic nature of components forming the mixture. However, the quasi-Bayesian estimation relies on mixing via constant...
Bayesian estimation of the mean holding time in average semi-Markov control processes
We consider semi-Markov control models with Borel state and action spaces, possibly unbounded costs, and holding times with a generalized exponential distribution with unknown mean θ. Assuming that such a distribution does not depend on the state-action pairs, we introduce a Bayesian estimation procedure for θ, which combined with a variant of the vanishing discount factor approach yields average cost optimal policies.
Bayesian parameter estimation and adaptive control of Markov processes with time-averaged cost
This paper considers Bayesian parameter estimation and an associated adaptive control scheme for controlled Markov chains and diffusions with time-averaged cost. Asymptotic behaviour of the posterior law of the parameter given the observed trajectory is analyzed. This analysis suggests a "cost-biased" estimation scheme and associated self-tuning adaptive control. This is shown to be asymptotically optimal in the almost sure sense.
Bellman approach to some problems in harmonic analysis
The stochastic optimal control uses the differential equation of Bellman and its solution - the Bellman function. Recently the Bellman function proved to be an efficient tool for solving some (sometimes old) problems in harmonic analysis.
Bibliography on stochastic approximation
Blind deconvolution of the aortic pressure waveform using the Malliavin calculus.
Boundary value problems arising in Kalman filtering.
Canonical input-output representation of linear multivariable stochastic systems and joint optimal parameter and state estimation.
In this paper a complete presentation is given of a new canonical representation of multi-input, multi-output linear stochastic systems. Its equivalence with operator form directly linked with ARMA processes as well as with classical state space representation is given, and a transfer matrix interpretation is developed in an example. The importance of the new representation is mainly in the fact that in the joint state and parameters estimation problem, all unknown parameters appear linearly when...
Causality and stochastic realization.
Chance constrained problems: penalty reformulation and performance of sample approximation technique
We explore reformulation of nonlinear stochastic programs with several joint chance constraints by stochastic programs with suitably chosen penalty-type objectives. We show that the two problems are asymptotically equivalent. Simpler cases with one chance constraint and particular penalty functions were studied in [6,11]. The obtained problems with penalties and with a fixed set of feasible solutions are simpler to solve and analyze then the chance constrained programs. We discuss solving both problems...
Characterization of a class of lifetime distributions
Characterization of the marginal distributions of Markov processes used in dynamic reliability.
Classical solutions of linear regulator for degenerate diffusions.
Classification of linear estimators
Closed-loop stability of discrete linear single-variable systems
Comparison principle approach to utility maximization
We consider the problem of optimal investment for maximal expected utility in an incomplete market with trading strategies subject to closed constraints. Under the assumption that the underlying utility function has constant sign, we employ the comparison principle for BSDEs to construct a family of supermartingales leading to a necessary and sufficient condition for optimality. As a consequence, the value function is characterized as the initial value of a BSDE with Lipschitz growth.