3
$\begingroup$

I'm trying to understand and learn more about "almost surely bounded" Markov chains on countable state spaces. I'm looking for references where I can learn how to work with more complicated examples of the following phenomenon.

The most simplified example is the Markov chain on $\mathbb{N}$ with transition probabilities $$p_{\rightarrow}(n) = 1/(1+n) \qquad p_\leftarrow(n) = n/(1+n)$$ I want to answer question about the stationary distribution like upper bounds for the probability of observing a large state. I think of this model as a spring: There is a constant force pushing to the right, and proportional force to the left.

In this simple example things can be worked out with elementary probability. The actual problem is in $\mathbb{N}^n$ with a finite number of transitions on each vertex, each with likelihoods. The likelihood to move out to infinity is constant and the likelihood to move back is proportional to the size of the state. I then want probabilities to stay in inside some polyhedron for long time.

To be specific: What are good references for Markov chains on $\mathbb{N}^n$ whose transitions are given by a finite set $E \subset \mathbb{Z}^n$, with probabilities for each transition $e\in E$ depending on the state $u\in\mathbb{N}^n$?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.