MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let us consider the homogeneous continuous time Markov chain $(X_t)_{t\ge 0}$ with two states {0,1} and the intensity matrix

$Q=\begin{pmatrix}-\lambda& \lambda\\\ \mu& -\mu\end{pmatrix}$

Let $N_t$ be the number of $1 \to 0$ transitions of $X_t$ in the interval [0, t].

The main interesting question is to find the probability $p_{ij}(k,t)=P(N_t=k, X_t= j| X_0=i)$.

May be we can use its generating functions $P_{ij}(z,t)=\sum_{k=0}^{\infty}z^kp_{i,j}(k,t)$ and recursion to obtain the result.

Have any ideas for a calculation?

share|cite|improve this question
up vote 2 down vote accepted

The usual infinitesimal analysis leading to a differential system applies, with a twist: for small $s>0$, $$ p_{i0}(k,t+s)=(1-\lambda s)p_{i0}(k,t)+\mu sp_{i1}(k-1,t)+o(s), $$ and $$ p_{i1}(k,t+s)=(1-\mu s)p_{i1}(k,t)+\lambda sp_{i0}(k,t)+o(s), $$ with the convention that $p_{ij}(-1,t)=0$ for every $t\geqslant0$. Hence, $$ \partial_tp_{i0}(k,t)=-\lambda p_{i0}(k,t)+\mu p_{i1}(k-1,t), $$ and $$ \partial_tp_{i1}(k,t)=-\mu p_{i1}(k,t)+\lambda p_{i0}(k,t). $$ With the initial condition $p_{ii}(0,0)=1$ for every $i$, $p_{ij}(0,0)=0$ for every $i\ne j$, and $p_{ij}(k,0)=0$ for every $(i,j)$ and every $k\geqslant1$, this determines uniquely $p_{ij}(k,t)$ for every $(i,j)$, $k\geqslant0$ and $t\geqslant0$.

share|cite|improve this answer

Let us consider a phase-type renewal process, where renewal intervals have $PH(\boldsymbol{\beta},S)$ distribution,

We set $S=\begin{pmatrix}-\lambda& \lambda\\\ 0 & -\mu\end{pmatrix}$ and $\boldsymbol{\beta}=(1,0)$

Denote by $P_{ij}(k,t)$ the probability that there occurs $k$ renewals in the interval [0,t] and the phase at $t$ is $j$ on condition that the phase at 0 is $i$.

Then it is well-known that the matrix $P(k,t)=(P_{ij}(k,t))$ can be defined by its generating function $$P(z,t)=\sum_{k=0}^{\infty}P(k,t)z^k=e^{G(z)t}$$ where $G(z) = S-S\boldsymbol{1}\boldsymbol{\beta}z= \begin{pmatrix}-\lambda& \lambda\\\ \mu z & -\mu\end{pmatrix} $.

Clearly, this probability is equivalent to the above-mentioned problem.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.