This paper deals with discrete-time Markov control processes in Borel spaces with unbounded rewards. Under suitable hypotheses, we show that a randomized stationary policy is optimal for a certain expected constrained problem (ECP) if and only if it is optimal for the corresponding pathwise constrained problem (pathwise CP). Moreover, we show that a certain parametric family of unconstrained optimality equations yields convergence properties that lead to an approximation scheme which allows us to...

This paper aims to provide a systematic approach to the treatment of differential equations of the typedyt = Σi fi(yt) dxti where the driving signal xt is a rough path. Such equations are very common and occur particularly frequently in probability where the driving signal might be a vector valued Brownian motion, semi-martingale or similar process.However, our approach is deterministic, is totally independent of probability and permits much rougher paths than the Brownian paths usually discussed....

This paper addresses a new differential game problem with forward-backward doubly stochastic differential equations. There are two distinguishing features. One is that our game systems are initial coupled, rather than terminal coupled. The other is that the admissible control is required to be adapted to a subset of the information generated by the underlying Brownian motions. We establish a necessary condition and a sufficient condition for an equilibrium point of nonzero-sum games and a saddle...

This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function,...

Two universally applicable smoothing operations adjustable to meet the specific properties of the given smoothing problem are widely used: 1. Smoothing splines and 2. Smoothing digital convolution filters. The first operation is related to the data vector $r={(r_0,...,r_{n-1})}^T$ with respect to the operations $𝒜$, $ℒ$ and to the smoothing parameter $\alpha$. The resulting function is denoted by ${\sigma }_{\alpha }\left(t\right)$. The measured sample $r$ is defined on an equally spaced mesh $\Delta ={\{t_i=ih\}}_{i=0}^{n-1}$$...$

Conditions for the absence of arbitrage in discrete time markets with various kinds of transaction costs are shown.

Long run risk sensitive portfolio selection is considered with proportional transaction costs. In the paper two methods to prove existence of solutions to suitable Bellman equations are presented. The first method is based on discounted cost approximation and requires uniform absolute continuity of iterations of transition operators of the factor process. The second method is based on uniform ergodicity of portions of the capital invested in assets and requires additional assumptions concerning...

Risk sensitive and risk neutral long run portfolio problems with consumption and proportional transaction costs are studied. Existence of solutions to suitable Bellman equations is shown. The asymptotics of the risk sensitive cost when the risk factor converges to 0 is then considered. It turns out that optimal strategies are stationary functions of the portfolio (portions of the wealth invested in assets) and of economic factors. Furthermore an optimal portfolio strategy for a risk neutral control...

This work analyzes a discrete-time Markov Control Model (MCM) on Borel spaces when the performance index is the expected total discounted cost. This criterion admits unbounded costs. It is assumed that the discount rate in any period is obtained by using recursive functions and a known initial discount rate. The classic dynamic programming method for finite-horizon case is verified. Under slight conditions, the existence of deterministic non-stationary optimal policies for infinite-horizon case...

Sensor networks are regarded as a promising technology in the field of information perception and processing owing to the ease of deployment, cost-effectiveness, flexibility, as well as reliability. The information exchange among sensors inevitably suffers from various network-induced phenomena caused by the limited resource utilization and complex application scenarios, and thus is required to be governed by suitable resource-saving communication mechanisms. It is also noteworthy that noises in...

In this paper, the distributed $H_{\infty }$ estimation problem is investigated for a moving target with local communication and switching topology. Based on the solution of the algebraic Riccati equation, a recursive algorithm is proposed using constant gain. The stability of the proposed algorithm is analysed by using the Lyapounov method, and a lower bound for estimation errors is obtained for the proposed common $H_{\infty }$ filter. Moreover, a bound for the $H_{\infty }$ parameter is obtained by means of the solution of the...

An approach to determine a scheduling policy for a sensor network monitoring some spatial domain in order to identify unknown parameters of a distributed system is discussed. Given a finite number of possible sites at which sensors are located, the activation schedule for scanning sensors is provided so as to maximize a criterion defined on the Fisher information matrix associated with the estimated parameters. The related combinatorial problem is relaxed through operating on the density of sensors...

This paper is concerned with the sampled-data based adaptive linear quadratic (LQ) control of hybrid systems with both unmeasurable Markov jump processes and stochastic noises. By the least matching error estimation algorithm, parameter estimates are presented. By a double-step (DS) sampling approach and the certainty equivalence principle, a sampled-data based adaptive LQ control is designed. The DS-approach is characterized by a comparatively large estimation step for parameter estimation and...

This paper is concerned with the sampled-data based adaptive linear quadratic (LQ) control of hybrid systems with both unmeasurable Markov jump processes and stochastic noises. By the least matching error estimation algorithm, parameter estimates are presented. By a double-step (DS) sampling approach and the certainty equivalence principle, a sampled-data based adaptive LQ control is designed. The DS-approach is characterized by a comparatively large estimation step for parameter estimation and...

We study the numerical approximation of doubly reflected backward stochastic differential equations with intermittent upper barrier (RIBSDEs). These denote reflected BSDEs in which the upper barrier is only active on certain random time intervals. From the point of view of financial interpretation, RIBSDEs arise as pricing equations of game options with constrained callability. In a Markovian set-up we prove a convergence rate for a time-discretization scheme by simulation to an RIBSDE. We also...