Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Let us consider the homogeneous continuous time Markov chain $(X_t)_{t\ge 0}$ with two states {0,1} and the intensity matrix

$Q=\begin{pmatrix}-\lambda& \lambda\\\ \mu& -\mu\end{pmatrix}$

Let $N_t$ be the number of $1 \to 0$ transitions of $X_t$ in the interval [0, t].

The main interesting question is to find the probability $p_{ij}(k,t)=P(N_t=k, X_t= j| X_0=i)$.

May be we can use its generating functions $P_{ij}(z,t)=\sum_{k=0}^{\infty}z^kp_{i,j}(k,t)$ and recursion to obtain the result.

Have any ideas for a calculation?

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

The usual infinitesimal analysis leading to a differential system applies, with a twist: for small $s>0$, $$ p_{i0}(k,t+s)=(1-\lambda s)p_{i0}(k,t)+\mu sp_{i1}(k-1,t)+o(s), $$ and $$ p_{i1}(k,t+s)=(1-\mu s)p_{i1}(k,t)+\lambda sp_{i0}(k,t)+o(s), $$ with the convention that $p_{ij}(-1,t)=0$ for every $t\geqslant0$. Hence, $$ \partial_tp_{i0}(k,t)=-\lambda p_{i0}(k,t)+\mu p_{i1}(k-1,t), $$ and $$ \partial_tp_{i1}(k,t)=-\mu p_{i1}(k,t)+\lambda p_{i0}(k,t). $$ With the initial condition $p_{ii}(0,0)=1$ for every $i$, $p_{ij}(0,0)=0$ for every $i\ne j$, and $p_{ij}(k,0)=0$ for every $(i,j)$ and every $k\geqslant1$, this determines uniquely $p_{ij}(k,t)$ for every $(i,j)$, $k\geqslant0$ and $t\geqslant0$.

share|improve this answer
add comment

Let us consider a phase-type renewal process, where renewal intervals have $PH(\boldsymbol{\beta},S)$ distribution, http://en.wikipedia.org/wiki/Phase-type_distribution

We set $S=\begin{pmatrix}-\lambda& \lambda\\\ 0 & -\mu\end{pmatrix}$ and $\boldsymbol{\beta}=(1,0)$

Denote by $P_{ij}(k,t)$ the probability that there occurs $k$ renewals in the interval [0,t] and the phase at $t$ is $j$ on condition that the phase at 0 is $i$.

Then it is well-known that the matrix $P(k,t)=(P_{ij}(k,t))$ can be defined by its generating function $$P(z,t)=\sum_{k=0}^{\infty}P(k,t)z^k=e^{G(z)t}$$ where $G(z) = S-S\boldsymbol{1}\boldsymbol{\beta}z= \begin{pmatrix}-\lambda& \lambda\\\ \mu z & -\mu\end{pmatrix} $.

Clearly, this probability is equivalent to the above-mentioned problem.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.