2
$\begingroup$

I know that for a continuous-time Markov chain, the probability of transition from time $0$ to $t$ is given by $P(t)=e^{Q(t)t}$. I have a system of $N$ interdependent continuous-time Markov chains evolving simultaneously. Each of the $N$ Markov chains can be in state 0 or 1 and the state of Markov chain $k$ at time $t$ is given by $X_k(t)$. The rate matrix of Markov chain $i$ is

$$Q_i=\left[ \begin{matrix} -\sum_{j=1}^N \lambda_{ji}I_{X_j(t)=1} & \sum_{j=1}^N \lambda_{ji}I_{X_j(t)=1}\\ \mu_i & -\mu_i \end{matrix}\right]$$

In such a coupled system of Markov chains, the forumula $P(X_i(t)=1\mid X_i(0)=0)$ is no longer the $(0,1)^{th}$ entry of $e^{Q_i t}$, since $Q_i$ in turn depends on the states in other Markov chains. How does one proceed to find $P(.)$ in such coupled Markov chain systems?

$\endgroup$
  • 1
    $\begingroup$ What you have is basically a contact process on a weighted graph, so you may find the theory of such processes useful (especially if you want to consider the limit $N\to\infty$). But, as lmg notes, for small $N$ you can just write down the explicit transition matrix for the combined process and exponentiate it. $\endgroup$ – Ilmari Karonen Apr 19 '14 at 4:38
2
$\begingroup$

Explicitly write down and exponentiate the rate matrix whose state space has size $2^N$.

For example this could describe the dynamics of Facebook membership among a group of $N$ people, where person $i$ has instantaneous rate $\mu_i$ of temporarily rage-quitting Facebook because they realize that Facebook is selling their entire activity stream to the NSA. But person $i$ is then enticed by the combined influence of the size $N-1$ group to re-join Facebook, which happens at an instantaneous rate proportional to the combined influence, where person $j$ has influence $\lambda_{ji}$ if person $j$ is currently on Facebook.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.