# How to simulate random paths of a non-homogeneous continuous-time Markov process with discrete state space for a given infinitesimal generator matrix?

1. Let $$X=(X_{t},\,t \in T)$$ be a non-homogeneous, continuous time Markov process with a finite state space $$S=\{1,...,K\}$$.

2. Let $$\alpha_{i,j}(t)$$ be the hazard rates of some $$\varGamma$$-distributed random variables.

My question: How can I simulate random paths of the Markov process with a transition intensity matrix which is built with the above hazard rates $$\alpha_{i,j}(t)$$?

For example: $$S=\{1,2,3,4\}$$ with given $$\alpha_{i,j}(t)$$ with $$i,j \in S$$.

Any idea is greatly appreciated.

## 1 Answer

Suppose you're in state $i$. For each $j$, let $X_j$ be an independent Exponential random variable with mean 1.

Now solve $\int_{0}^{T_j}\alpha_{i,j}(t)\ dt=X_j$ for each $i$. Whichever of the $T_j$'s is smallest, you jump from state $i$ to state $j$ at time $T_j$.

• Thank you very much for your answer. Do you have a reference (a book or paper) for your answer. I will now apply this procedure in MATHEMATICA. Again, thank you! – Wolfgang123 May 31 '13 at 5:44
• No sorry I don't. Something like this has been in the back of my mind for a while. You can check it though: compute the probability that there's a transition from $i$ to $j$ in the time interval $(t,t+dt)$ conditioned that there has been no transition up to time t. – Anthony Quas May 31 '13 at 6:04
• I tested it with constant transition intensities alpha and it works fine. For general alpha's as described above, it seems that it does not work correctly. Maybe I make a mistake. However, I will now apply your method to simulate a 3-state model (illness-death model). In this case, the analytical solution is known and will compare it with numerical results. – Wolfgang123 May 31 '13 at 8:19
• It works pretty fine with the reduced model. I generalized the model and solved the corresponding Chapman-Kolmogorov equation system and everything works pretty fine. Thank you VERY much, with best wishes Wolfgang – Wolfgang123 May 31 '13 at 10:29