Markov chain: Obtaining transition matrix from recurrence probabilities - MathOverflow

Markov chain: Obtaining transition matrix from recurrence probabilities

2011-03-09T13:19:52Z

Consider a markov chain with finite space { 0,1,..n} with transition probability matrix whose entries are $P_{ij}$. Let

$f_{ij}^n$ = probability that starting from state $i $ it goes to state $j$ first time.

Question 1. What are the necessary and sufficient condition arbitrary $a_{ij}^{n}$ needs to satisfy to be valid $f_{ij}^{n}$ of some markov chain ?

Qustion 2. If existence of such a markov chain is shown, can we calculate $P_{ij}$ given valid $f_{ij}^n$ ? ( I mean, is there any algorithm to calculate?)

Answer by Didier Piau for Markov chain: Obtaining transition matrix from recurrence probabilities

2011-03-09T13:34:56Z

Re 2, for $n=1$, $P_{ij}=f^1_{ij}$ for every $i\ne j$ and $1-P_{ii}$ is the sum of $f^1_{ij}$ over $j\ne i$, hence one recovers trivially $P$ from $f^1=(f^1_{ij})_{ij}$.

Re 2 again, on the contrary, there is no hope to recover $P$ from $f^n=(f^n_{ij})_{ij}$ in general, for any given $n\ge2$. For example, the two (deterministic) one-step rotations on the discrete circle with $2n$ vertices, clockwise and counterclockwide, yield the same matrix $f^n$ although their transition matrices $P$ are different.