Displaying similar documents to “Nonzero-sum semi-Markov games with countable state spaces”

Bi-personal stochastic transient Markov games with stopping times and total reward criterion

Martínez-Cortés Victor Manuel (2021)

Kybernetika

Similarity:

The article is devoted to a class of Bi-personal (players 1 and 2), zero-sum Markov games evolving in discrete-time on Transient Markov reward chains. At each decision time the second player can stop the system by paying terminal reward to the first player. If the system is not stopped the first player selects a decision and two things will happen: The Markov chain reaches next state according to the known transition law, and the second player must pay a reward to the first player. The...

Some remarks on equilibria in semi-Markov games

Andrzej Nowak (2000)

Applicationes Mathematicae

Similarity:

This paper is a first study of correlated equilibria in nonzero-sum semi-Markov stochastic games. We consider the expected average payoff criterion under a strong ergodicity assumption on the transition structure of the games. The main result is an extension of the correlated equilibrium theorem proven for discounted (discrete-time) Markov games in our joint paper with Raghavan. We also provide an existence result for stationary Nash equilibria in the limiting average payoff semi-Markov...

Bottom-up modeling of domestic appliances with Markov chains and semi-Markov processes

Rajmund Drenyovszki, Lóránt Kovács, Kálmán Tornai, András Oláh, István Pintér (2017)

Kybernetika

Similarity:

In our paper we investigate the applicability of independent and identically distributed random sequences, first order Markov and higher order Markov chains as well as semi-Markov processes for bottom-up electricity load modeling. We use appliance time series from publicly available data sets containing fine grained power measurements. The comparison of models are based on metrics which are supposed to be important in power systems like Load Factor, Loss of Load Probability. Furthermore,...

Markov stopping games with an absorbing state and total reward criterion

Rolando Cavazos-Cadena, Luis Rodríguez-Gutiérrez, Dulce María Sánchez-Guillermo (2021)

Kybernetika

Similarity:

This work is concerned with discrete-time zero-sum games with Markov transitions on a denumerable space. At each decision time player II can stop the system paying a terminal reward to player I, or can let the system to continue its evolution. If the system is not halted, player I selects an action which affects the transitions and receives a running reward from player II. Assuming the existence of an absorbing state which is accessible from any other state, the performance of a pair...

Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games

Kristal K. Trejo, Julio B. Clempner, Alexander S. Poznyak (2015)

International Journal of Applied Mathematics and Computer Science

Similarity:

In this paper we present the extraproximal method for computing the Stackelberg/Nash equilibria in a class of ergodic controlled finite Markov chains games. We exemplify the original game formulation in terms of coupled nonlinear programming problems implementing the Lagrange principle. In addition, Tikhonov's regularization method is employed to ensure the convergence of the cost-functions to a Stackelberg/Nash equilibrium point. Then, we transform the problem into a system of equations...

Single-use reliability computation of a semi-Markovian system

Guglielmo D'Amico (2014)

Applications of Mathematics

Similarity:

Markov chain usage models were successfully used to model systems and software. The most prominent approaches are the so-called failure state models Whittaker and Thomason (1994) and the arc-based Bayesian models Sayre and Poore (2000). In this paper we propose arc-based semi-Markov usage models to test systems. We extend previous studies that rely on the Markov chain assumption to the more general semi-Markovian setting. Among the obtained results we give a closed form representation...

Handling a Kullback-Leibler divergence random walk for scheduling effective patrol strategies in Stackelberg security games

César U. S. Solis, Julio B. Clempner, Alexander S. Poznyak (2019)

Kybernetika

Similarity:

This paper presents a new model for computing optimal randomized security policies in non-cooperative Stackelberg Security Games (SSGs) for multiple players. Our framework rests upon the extraproximal method and its extension to Markov chains, within which we explicitly compute the unique Stackelberg/Nash equilibrium of the game by employing the Lagrange method and introducing the Tikhonov regularization method. We also consider a game-theory realization of the problem that involves...

The expected cumulative operational time for finite semi-Markov systems and estimation

Brahim Ouhbi, Ali Boudi, Mohamed Tkiouat (2007)

RAIRO - Operations Research

Similarity:

In this paper we, firstly, present a recursive formula of the empirical estimator of the semi-Markov kernel. Then a non-parametric estimator of the expected cumulative operational time for semi-Markov systems is proposed. The asymptotic properties of this estimator, as the uniform strongly consistency and normality are given. As an illustration example, we give a numerical application.

A generalization of Ueno's inequality for n-step transition probabilities

Andrzej Nowak (1998)

Applicationes Mathematicae

Similarity:

We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.