2
$\begingroup$

I know that for a continuous-time Markov chain, the probability of transition from time $0$ to $t$ is given by $P(t)=e^{Q(t)t}$. I have a system of $N$ interdependent continuous-time Markov chains evolving simultaneously. Each of the $N$ Markov chains can be in state 0 or 1 and the state of Markov chain $k$ at time $t$ is given by $X_k(t)$. The rate matrix of Markov chain $i$ is

$$Q_i=\left[ \begin{matrix} -\sum_{j=1}^N \lambda_{ji}I_{X_j(t)=1} & \sum_{j=1}^N \lambda_{ji}I_{X_j(t)=1}\\ \mu_i & -\mu_i \end{matrix}\right]$$

In such a coupled system of Markov chains, the forumula $P(X_i(t)=1\mid X_i(0)=0)$ is no longer the $(0,1)^{th}$ entry of $e^{Q_i t}$, since $Q_i$ in turn depends on the states in other Markov chains. How does one proceed to find $P(.)$ in such coupled Markov chain systems?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ What you have is basically a contact process on a weighted graph, so you may find the theory of such processes useful (especially if you want to consider the limit $N\to\infty$). But, as lmg notes, for small $N$ you can just write down the explicit transition matrix for the combined process and exponentiate it. $\endgroup$
    – Ilmari Karonen
    Commented Apr 19, 2014 at 4:38

1 Answer 1

Reset to default
2
$\begingroup$

Explicitly write down and exponentiate the rate matrix whose state space has size $2^N$.

For example this could describe the dynamics of Facebook membership among a group of $N$ people, where person $i$ has instantaneous rate $\mu_i$ of temporarily rage-quitting Facebook because they realize that Facebook is selling their entire activity stream to the NSA. But person $i$ is then enticed by the combined influence of the size $N-1$ group to re-join Facebook, which happens at an instantaneous rate proportional to the combined influence, where person $j$ has influence $\lambda_{ji}$ if person $j$ is currently on Facebook.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .