0
$\begingroup$

Let us consider the homogeneous continuous time Markov chain $(X_t)_{t\ge 0}$ with two states {0,1} and the intensity matrix

$Q=\begin{pmatrix}-\lambda& \lambda\\\ \mu& -\mu\end{pmatrix}$

Let $N_t$ be the number of $1 \to 0$ transitions of $X_t$ in the interval [0, t].

The main interesting question is to find the probability $p_{ij}(k,t)=P(N_t=k, X_t= j| X_0=i)$.

May be we can use its generating functions $P_{ij}(z,t)=\sum_{k=0}^{\infty}z^kp_{i,j}(k,t)$ and recursion to obtain the result.

Have any ideas for a calculation?

$\endgroup$

2 Answers 2

3
$\begingroup$

The usual infinitesimal analysis leading to a differential system applies, with a twist: for small $s>0$, $$ p_{i0}(k,t+s)=(1-\lambda s)p_{i0}(k,t)+\mu sp_{i1}(k-1,t)+o(s), $$ and $$ p_{i1}(k,t+s)=(1-\mu s)p_{i1}(k,t)+\lambda sp_{i0}(k,t)+o(s), $$ with the convention that $p_{ij}(-1,t)=0$ for every $t\geqslant0$. Hence, $$ \partial_tp_{i0}(k,t)=-\lambda p_{i0}(k,t)+\mu p_{i1}(k-1,t), $$ and $$ \partial_tp_{i1}(k,t)=-\mu p_{i1}(k,t)+\lambda p_{i0}(k,t). $$ With the initial condition $p_{ii}(0,0)=1$ for every $i$, $p_{ij}(0,0)=0$ for every $i\ne j$, and $p_{ij}(k,0)=0$ for every $(i,j)$ and every $k\geqslant1$, this determines uniquely $p_{ij}(k,t)$ for every $(i,j)$, $k\geqslant0$ and $t\geqslant0$.

$\endgroup$
0
$\begingroup$

Let us consider a phase-type renewal process, where renewal intervals have $PH(\boldsymbol{\beta},S)$ distribution, http://en.wikipedia.org/wiki/Phase-type_distribution

We set $S=\begin{pmatrix}-\lambda& \lambda\\\ 0 & -\mu\end{pmatrix}$ and $\boldsymbol{\beta}=(1,0)$

Denote by $P_{ij}(k,t)$ the probability that there occurs $k$ renewals in the interval [0,t] and the phase at $t$ is $j$ on condition that the phase at 0 is $i$.

Then it is well-known that the matrix $P(k,t)=(P_{ij}(k,t))$ can be defined by its generating function $$P(z,t)=\sum_{k=0}^{\infty}P(k,t)z^k=e^{G(z)t}$$ where $G(z) = S-S\boldsymbol{1}\boldsymbol{\beta}z= \begin{pmatrix}-\lambda& \lambda\\\ \mu z & -\mu\end{pmatrix} $.

Clearly, this probability is equivalent to the above-mentioned problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.