# Are interarrival times of doubly-stochastic Poisson I.I.D.?

I am working on a Markov-modulated Poisson process $\{N_{t}, t \geq 0\}$, which is itself a Poisson but the rates of which are governed by a CTMC. In my case, the CTMC is a one-class, aperiodic and positive recurrent MC.

My questions are the following

1. Are the interarrival times of $\{N_t, t \geq 0\}$ i.i.d? My guess is NO, and it is because the rate of the second interarrival is dependent on the state of the previous one.

2. If I want to get the distributions of interarrival times, say $X_1$, can I do $$\mathbb{P}\{X_1 > t\} = \sum_{s \in \mathcal{S}}e^{-\mu_{s}t}\pi_{s}$$ where $\pi_{s}$ is a stationary initial distribution for state $s$ in the CTMC. Is this correct?

3. If I want further to compute the joint distribution of interarrival times, I do the following $$\mathbb{P}\{X_1 > t,...,X_n > t\} = \sum_{s_1, s_2,...,s_n}\prod_{k=1}^{n}e^{-\mu_{s_k}t}\prod_{l=1}^{n-1}R_{s_l,s_{l+1}}\pi_{s_1}$$ For instance, $$\mathbb{P}\{X_1 > t, X_2 > t, X_3 > t\} = \sum_{s_1 \in \mathcal{S}, s_2 \in S_1, s_3 \in S_2}e^{-\mu_{s_1}t}e^{-\mu_{s_2}t}e^{-\mu_{s_3}t}R_{s_1,s_2}R_{s_2,s_3}\pi_{s_1}$$ where $S_1$ is the possible next states connected to $s_1$; $R_{s_1,s_2}$ is the transition probability from $s_1$ to $s_2$; and $\pi_{s_1}$ is a stationary initial distribution for the CTMC. What I do here is simply conditioning on all possible states for interarrival times and use the independence after conditioning. But I am not quite sure about this.

Motivation

Since there has been no reply to the above questions, which may be because they looked like homework questions. But it is not. As a matter of fact, I am trying to solve the following random sum $$Q = \sum_{i=1}^{N}X_i$$ wherein $X_i$ is the i-th interarrival time of an MMPP and $N$ is also a random variable defined as $$N = \inf\{n : X_1 < P,...,X_n < P, X_{n+1} > P\}$$ namely the first time we have an interarrival time greater than some constant $P$. My goal is to get distributional properties of the random sum (distribution, expectation and so like).

• Can you please clarify what you understand by the interarrival time $X_n$? Your formulae are correct if these describe changes of the underlying continuous time Markov chain. However, you suggest that there is another Poisson-type process that is governed by the Markov chain, so I guess that $X_n$ are waiting times for this Poisson-type process. (It would be much easier if you gave these processes names, say: $S_t$ for the CTMC, $N_t$ for the Poisson-type counting process). Sep 10 '17 at 13:17
• Thanks Mateusz. I have only one process which is an MMPP $\{N_{t}, t \geq 0\}$ (a Poisson process whose rate is governed by a continuous time Markov chain, with infinitesimal generator $\mathbf{Q}$ and transition probability matrix $R$). And I want to know the distributional properties of the time between successive events generated by this underlying MMPP. (hopefully I have made clear this time)
– Liäm
Sep 10 '17 at 13:33

If I understand correctly, there is a hidden Markov chain, say $(S_t)$, whose state $s = S_t$ describes the rate $\mu_s$ at which signals of the observed counting process $N_t$ arrive. If this is correct, then the distribution of the waiting time for the first signal is much more complicated. The best way to understand it is to see how $N_t$ is constructed.

To avoid nested indices, let us use the notation $N(t)$, $\mu(s)$ etc. First, define the additive functional $$A(t) = \int_0^t \mu(S(\tau)) d\tau .$$ This functional measures the average number of signals up to time $t$, conditionally on a fixed path of $(S(t))$. Therefore, if $(\tilde{N}(t))$ is a Poisson process with intensity $1$, then $(N(t))$ can be defined as $$N(t) = \tilde{N}(A(t)).$$ Therefore, the waiting time for the first signal $X_1$ satisfies $$\mathbb{P}(X_1>t) = \mathbb{P}(\tilde{N}(A(t)) = 0) .$$ Using the independence of $(\tilde{N}(t))$ and $(A(t))$, we get $$\mathbb{P}(X_1>t) = \mathbb{E} e^{-A(t)} .$$ The quantity $e^{-A(t)}$ is a multiplicative functional. Evaluation of its expectation $$u(t, s) = \mathbb{E}(e^{-A(t)} | S(0) = s)$$ for an arbitrary Markov process $(S(t))$ is a non-trivial problem: $u$ solves the Schrödinger evolution equation $$\partial_t u(t, s) = L u(t, s) - \mu(s) u(t, s),$$ where $L$ is the generator of $(S(t))$. For a continuous time Markov chain on a finite state space $\{s_1, s_2, \ldots, s_n\}$ this reduces to a system of ODEs: the vector $u(t) = (u(t, s_1), u(t, s_2), \ldots, u(t, s_n))$ satisfies $$u'(t) = \mathbb{Q} u(t) - \mu u(t),$$ where $\mathbb{Q}$ is the generator matrix and $\mu$ is a diagonal matrix with entries $\mu(s_1), \mu(s_2), \ldots, \mu(s_n)$ on the diagonal; thus $u(t) = \exp(t (L - \mu))$.

Starting from a stationary distribution of $(S(t))$ does not simplify the above equation. Furthermore, after the first signal arrives, $(S(t))$ need not be in a stationary distribution any more! This makes the evaluation of the waiting time for the second signal $X_2$ apparently complicated.

Let me finish with two simple examples. First, suppose that $(S(t))$ has two states $s_1$ and $s_2$, with $\mu(s_1) = 0$ and $\mu(s_2) = 1$, and that the transition rate between $s_1$ and $s_2$ is $1$ in both directions. The stationary distribution is $\mathbb{P}(S_0 = s_1) = \mathbb{P}(S_0 = s_2) = \tfrac{1}{2}$. After the first signal, $(S(t))$ is necessarily in state $s_2$. In particular, this means that $X_1$ and $X_2$ have different distributions!

Now suppose that $(S(t))$ again has two states $s_1$ and $s_2$, but this time $\mu(s_1) = 1$ and $\mu(s_2) = 1000$, and the transition rate between $s_1$ and $s_2$ is zero (or just very small if you want $(S(t))$ to be ergodic) in both directions. Again $\mathbb{P}(S_0 = s_1) = \mathbb{P}(S_0 = s_2) = \tfrac{1}{2}$ is a stationary distribution. If $X_1$ is small, it is very likely that $(S(t))$ started in state $s_2$, and thus $X_2$ is likely to be small as well. On the other hand, if $X_1$ is of the order of $1$, then we can expect that $(S(t))$ is in state $s_1$, and so $X_2$ is highly unlikely to be small. (Remember that $(S(t))$ does not change state at all, or at least it does change it with very low intensity). Therefore, $X_1$ and $X_2$ are not independent.

• Thanks Mateusz for the clear elaboration! Your terminologies (like the Schrodinger thing) are completely beyond my knowledge, but I get what you meant. So if I understand correctly, those $X_i$'s are conditionally i.i.d (condition on the current state for each $X_i$) and what I did was legit in this sense
– Liäm
Sep 11 '17 at 12:06
• Yes, that's correct: $X_i$ depends on the past only through the state of $S_t$ at $t = X_1+X_2+\ldots+X_{i-1}$. Sep 11 '17 at 12:26