Necessary and sufficient optimality conditions for average reward of controlled Markov chains
Non-colliding random walks, tandem queues, and discrete orthogonal polynomial ensembles.
Non-Homogeneous Markov Chains: Tail Idempotents, Tail Sigma-Fields and Basis.
Non-homogeneous Markov chains with a finite state space and a Doeblin type theorem.
Nonlinear Markov processes in big networks
Big networks express multiple classes of large-scale networks in many practical areas such as computer networks, internet of things, cloud computation, manufacturing systems, transportation networks, and healthcare systems. This paper analyzes such big networks, and applies the mean-field theory and the nonlinear Markov processes to constructing a broad class of nonlinear continuous-time block-structured Markov processes, which can be used to deal with many practical stochastic systems. Firstly,...
Note: random-to-front shuffles on trees.
Number of hidden states and memory: a joint order estimation problem for Markov chains with Markov regime
This paper deals with order identification for Markov chains with Markov regime (MCMR) in the context of finite alphabets. We define the joint order of a MCMR process in terms of the number k of states of the hidden Markov chain and the memory m of the conditional Markov chain. We study the properties of penalized maximum likelihood estimators for the unknown order (k, m) of an observed MCMR process, relying on information theoretic arguments. The novelty of our work relies in the joint...