I am working on a Markov-modulated Poisson process $\{N_{t}, t \geq 0\}$, which is itself a Poisson but the rates of which are governed by a CTMC. In my case, the CTMC is a one-class, aperiodic and positive recurrent MC.

My questions are the following

  1. Are the interarrival times of $\{N_t, t \geq 0\}$ i.i.d? My guess is NO, and it is because the rate of the second interarrival is dependent on the state of the previous one.

  2. If I want to get the distributions of interarrival times, say $X_1$, can I do \begin{equation} \mathbb{P}\{X_1 > t\} = \sum_{s \in \mathcal{S}}e^{-\mu_{s}t}\pi_{s} \end{equation} where $\pi_{s}$ is a stationary initial distribution for state $s$ in the CTMC. Is this correct?

  3. If I want further to compute the joint distribution of interarrival times, I do the following \begin{equation} \mathbb{P}\{X_1 > t,...,X_n > t\} = \sum_{s_1, s_2,...,s_n}\prod_{k=1}^{n}e^{-\mu_{s_k}t}\prod_{l=1}^{n-1}R_{s_l,s_{l+1}}\pi_{s_1} \end{equation} For instance, \begin{equation} \mathbb{P}\{X_1 > t, X_2 > t, X_3 > t\} = \sum_{s_1 \in \mathcal{S}, s_2 \in S_1, s_3 \in S_2}e^{-\mu_{s_1}t}e^{-\mu_{s_2}t}e^{-\mu_{s_3}t}R_{s_1,s_2}R_{s_2,s_3}\pi_{s_1} \end{equation} where $S_1$ is the possible next states connected to $s_1$; $R_{s_1,s_2}$ is the transition probability from $s_1$ to $s_2$; and $\pi_{s_1}$ is a stationary initial distribution for the CTMC. What I do here is simply conditioning on all possible states for interarrival times and use the independence after conditioning. But I am not quite sure about this.


Since there has been no reply to the above questions, which may be because they looked like homework questions. But it is not. As a matter of fact, I am trying to solve the following random sum \begin{equation} Q = \sum_{i=1}^{N}X_i \end{equation} wherein $X_i$ is the i-th interarrival time of an MMPP and $N$ is also a random variable defined as \begin{equation} N = \inf\{n : X_1 < P,...,X_n < P, X_{n+1} > P\} \end{equation} namely the first time we have an interarrival time greater than some constant $P$. My goal is to get distributional properties of the random sum (distribution, expectation and so like).

| cite | improve this question | | | | |
  • $\begingroup$ Can you please clarify what you understand by the interarrival time $X_n$? Your formulae are correct if these describe changes of the underlying continuous time Markov chain. However, you suggest that there is another Poisson-type process that is governed by the Markov chain, so I guess that $X_n$ are waiting times for this Poisson-type process. (It would be much easier if you gave these processes names, say: $S_t$ for the CTMC, $N_t$ for the Poisson-type counting process). $\endgroup$ – Mateusz Kwaśnicki Sep 10 '17 at 13:17
  • $\begingroup$ Thanks Mateusz. I have only one process which is an MMPP $\{N_{t}, t \geq 0\}$ (a Poisson process whose rate is governed by a continuous time Markov chain, with infinitesimal generator $\mathbf{Q}$ and transition probability matrix $R$). And I want to know the distributional properties of the time between successive events generated by this underlying MMPP. (hopefully I have made clear this time) $\endgroup$ – Liäm Sep 10 '17 at 13:33

If I understand correctly, there is a hidden Markov chain, say $(S_t)$, whose state $s = S_t$ describes the rate $\mu_s$ at which signals of the observed counting process $N_t$ arrive. If this is correct, then the distribution of the waiting time for the first signal is much more complicated. The best way to understand it is to see how $N_t$ is constructed.

To avoid nested indices, let us use the notation $N(t)$, $\mu(s)$ etc. First, define the additive functional $$ A(t) = \int_0^t \mu(S(\tau)) d\tau .$$ This functional measures the average number of signals up to time $t$, conditionally on a fixed path of $(S(t))$. Therefore, if $(\tilde{N}(t))$ is a Poisson process with intensity $1$, then $(N(t))$ can be defined as $$ N(t) = \tilde{N}(A(t)). $$ Therefore, the waiting time for the first signal $X_1$ satisfies $$ \mathbb{P}(X_1>t) = \mathbb{P}(\tilde{N}(A(t)) = 0) . $$ Using the independence of $(\tilde{N}(t))$ and $(A(t))$, we get $$ \mathbb{P}(X_1>t) = \mathbb{E} e^{-A(t)} . $$ The quantity $e^{-A(t)}$ is a multiplicative functional. Evaluation of its expectation $$u(t, s) = \mathbb{E}(e^{-A(t)} | S(0) = s)$$ for an arbitrary Markov process $(S(t))$ is a non-trivial problem: $u$ solves the Schrödinger evolution equation $$\partial_t u(t, s) = L u(t, s) - \mu(s) u(t, s),$$ where $L$ is the generator of $(S(t))$. For a continuous time Markov chain on a finite state space $\{s_1, s_2, \ldots, s_n\}$ this reduces to a system of ODEs: the vector $u(t) = (u(t, s_1), u(t, s_2), \ldots, u(t, s_n))$ satisfies $$u'(t) = \mathbb{Q} u(t) - \mu u(t),$$ where $\mathbb{Q}$ is the generator matrix and $\mu$ is a diagonal matrix with entries $\mu(s_1), \mu(s_2), \ldots, \mu(s_n)$ on the diagonal; thus $u(t) = \exp(t (L - \mu))$.

Starting from a stationary distribution of $(S(t))$ does not simplify the above equation. Furthermore, after the first signal arrives, $(S(t))$ need not be in a stationary distribution any more! This makes the evaluation of the waiting time for the second signal $X_2$ apparently complicated.

Let me finish with two simple examples. First, suppose that $(S(t))$ has two states $s_1$ and $s_2$, with $\mu(s_1) = 0$ and $\mu(s_2) = 1$, and that the transition rate between $s_1$ and $s_2$ is $1$ in both directions. The stationary distribution is $\mathbb{P}(S_0 = s_1) = \mathbb{P}(S_0 = s_2) = \tfrac{1}{2}$. After the first signal, $(S(t))$ is necessarily in state $s_2$. In particular, this means that $X_1$ and $X_2$ have different distributions!

Now suppose that $(S(t))$ again has two states $s_1$ and $s_2$, but this time $\mu(s_1) = 1$ and $\mu(s_2) = 1000$, and the transition rate between $s_1$ and $s_2$ is zero (or just very small if you want $(S(t))$ to be ergodic) in both directions. Again $\mathbb{P}(S_0 = s_1) = \mathbb{P}(S_0 = s_2) = \tfrac{1}{2}$ is a stationary distribution. If $X_1$ is small, it is very likely that $(S(t))$ started in state $s_2$, and thus $X_2$ is likely to be small as well. On the other hand, if $X_1$ is of the order of $1$, then we can expect that $(S(t))$ is in state $s_1$, and so $X_2$ is highly unlikely to be small. (Remember that $(S(t))$ does not change state at all, or at least it does change it with very low intensity). Therefore, $X_1$ and $X_2$ are not independent.

| cite | improve this answer | | | | |
  • $\begingroup$ Thanks Mateusz for the clear elaboration! Your terminologies (like the Schrodinger thing) are completely beyond my knowledge, but I get what you meant. So if I understand correctly, those $X_i$'s are conditionally i.i.d (condition on the current state for each $X_i$) and what I did was legit in this sense $\endgroup$ – Liäm Sep 11 '17 at 12:06
  • $\begingroup$ Yes, that's correct: $X_i$ depends on the past only through the state of $S_t$ at $t = X_1+X_2+\ldots+X_{i-1}$. $\endgroup$ – Mateusz Kwaśnicki Sep 11 '17 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.