Number of transitions of a markov chain in a time interval

Question

Let us consider the homogeneous continuous time Markov chain $(X_t)_{t\ge 0}$ with two states {0,1} and the intensity matrix

$Q=\begin{pmatrix}-\lambda& \lambda\\\ \mu& -\mu\end{pmatrix}$

Let $N_t$ be the number of $1 \to 0$ transitions of $X_t$ in the interval [0, t].

The main interesting question is to find the probability $p_{ij}(k,t)=P(N_t=k, X_t= j| X_0=i)$.

May be we can use its generating functions $P_{ij}(z,t)=\sum_{k=0}^{\infty}z^kp_{i,j}(k,t)$ and recursion to obtain the result.

Have any ideas for a calculation?

Answer 1

The usual infinitesimal analysis leading to a differential system applies, with a twist: for small $s>0$, $$ p_{i0}(k,t+s)=(1-\lambda s)p_{i0}(k,t)+\mu sp_{i1}(k-1,t)+o(s), $$ and $$ p_{i1}(k,t+s)=(1-\mu s)p_{i1}(k,t)+\lambda sp_{i0}(k,t)+o(s), $$ with the convention that $p_{ij}(-1,t)=0$ for every $t\geqslant0$. Hence, $$ \partial_tp_{i0}(k,t)=-\lambda p_{i0}(k,t)+\mu p_{i1}(k-1,t), $$ and $$ \partial_tp_{i1}(k,t)=-\mu p_{i1}(k,t)+\lambda p_{i0}(k,t). $$ With the initial condition $p_{ii}(0,0)=1$ for every $i$, $p_{ij}(0,0)=0$ for every $i\ne j$, and $p_{ij}(k,0)=0$ for every $(i,j)$ and every $k\geqslant1$, this determines uniquely $p_{ij}(k,t)$ for every $(i,j)$, $k\geqslant0$ and $t\geqslant0$.

Answer 2

Let us consider a phase-type renewal process, where renewal intervals have $PH(\boldsymbol{\beta},S)$ distribution, http://en.wikipedia.org/wiki/Phase-type_distribution

We set $S=\begin{pmatrix}-\lambda& \lambda\\\ 0 & -\mu\end{pmatrix}$ and $\boldsymbol{\beta}=(1,0)$

Denote by $P_{ij}(k,t)$ the probability that there occurs $k$ renewals in the interval [0,t] and the phase at $t$ is $j$ on condition that the phase at 0 is $i$.

Then it is well-known that the matrix $P(k,t)=(P_{ij}(k,t))$ can be defined by its generating function $$P(z,t)=\sum_{k=0}^{\infty}P(k,t)z^k=e^{G(z)t}$$ where $G(z) = S-S\boldsymbol{1}\boldsymbol{\beta}z= \begin{pmatrix}-\lambda& \lambda\\\ \mu z & -\mu\end{pmatrix} $.

Clearly, this probability is equivalent to the above-mentioned problem.