Occupation times for two-state Markov processes

Question

Consider a two-state Markov process in continuous time, with states labelled $A$ and $B$. The transition rates for going from state $A$ to $B$, and state $B$ to $A$ are $\alpha$ and $\beta$ respectively. We are interested in the quantity $p_t(A,x,t_A|A)$, which denotes the probability that, at time $t$, the Markov process is in state $A$, has made a total of $x$ jumps, and has spent time equal to $t_A$ in state $A$, given that it started from state $A$.

I was thinking of two approaches (and both are likely to be equivalent). But I seem to be running into hurdles in both of them, and would be grateful for some help.

My first approach was to write a set of coupled recurrence relations in the following way: \begin{align} p_t(A,x,t_A|A) &= \int_0^t e^{-\alpha t'}\cdot \alpha \cdot p_{t-t'}(A,x-1,t_A-t'|B)\cdot dt'\\ p_t(A,x,t_A|B) &= \int_0^t e^{-\beta t'}\cdot \beta \cdot p_{t-t'}(A,x-1,t_A|A)\cdot dt' \end{align} The basic idea behind the above equations is that the first jump takes place at time $t'$, and thus, the rest of the $x-1$ jumps have to be made in the remaining $t-t'$ amount of time. Similarly, depending on what the initial state was, the time needed to be spent in state $A$ in the remaining $t-t'$ time is stated in the argument (e.g. if the first $t'$ amount of time was spent in state $A$, then in the remaining amount of time, only $t_A-t'$ amount of time needs to be spent there). There is a minor issue here that the above equation does not necessarily restrict the possibility of $t'>t_A$, which is unphysical. But that can be taken care of by changing the upper limit of the integral in the first equation to $t_A$.

Naively, one would expect that taking a Laplace transform of the two equations with respect to $t$ should simplify matters a lot. For the second equation, it indeed does, as the equation is a simple convolution in $t$. However, the first is not (please correct me if I am wrong) and thus, naive Laplace transform with respect to $t$ might not be helpful. Is there any other approach which makes the problem easier?

My second approach was to derive the differential equation satisfied by the quantity $p_t(A,x,t_A|A)$, by working in discrete time-steps of $\Delta t$, and finally taking the limit $\Delta t \to 0$:

\begin{align} p_{t+\Delta t}(A,x,t_A|A) = p_t(A,x,t_A-\Delta t|A)\cdot (1-\alpha \Delta t) + p_t(B,x-1,t_A-\Delta t|A)\cdot \beta \Delta t \end{align} which upon rearranging gives us \begin{align} \frac{p_{t+\Delta t}(A,x,t_A|A) - p_{t}(A,x,t_A- \Delta t|A)}{\Delta t} = -\alpha\cdot p_t(A,x,t_A-\Delta t|A) + \beta \cdot p_t(B,x-1,t_A - \Delta t|A). \end{align} In the limit of $\Delta t \to 0$, we get: $$\frac{\partial{p_{t}(A,x,t_A|A)}}{\partial{t}}+\frac{\partial{p_{t}(A,x,t_A|A)}}{\partial{t_A}} = -\alpha\cdot p_t(A,x,t_A|A) + \beta \cdot p_t(B,x-1,t_A|A). $$

Similarly, one could write equations for $p_t(B,x,t_A|A)$. Here (again naively), it seems like one could take a Laplace transform with respect to $t$ and $t_A$, and define a generating function for $x$ to simplify the problem. However, one thing that scares me is that $t_A$ cannot take any value. It must always take values less than $t$. Thus, independently taking Laplace transforms with respect to $t$ and $t_A$ seems shady.

Is there a way to overcome one (or both) of the hurdles?

Answer 1

Without loss of generality, let the final time to be $t=1$ (if it is not, we can make it so by rescaling time as $\alpha'=t\alpha$ and $\beta'=t\alpha$).

Then, consider a single trajectory of the process that starts and ends on state $A$ and makes $x$ jumps ($x$ has to be even). We can represent any such trajectory in terms of the jump times, $$\vec{t}=(t_{0}=0\le t_{1}\le\dots\le t_{x}\le1) \tag{1}$$ The conditional probability of the trajectory is given by the product of the jump probabilities, multiplied by the probability of not leaving state $A$ after the last jump: $$p(\vec{t}\vert A)=e^{-\alpha(1-t_{x})}\prod_{i=0}^{x-1}p_{i}(t_{i+1}\vert t_{i}).\tag{2}$$ We will assume below that $x\ge 2$ (otherwise $t_A=1$ and $p(A,x,t_A|A)=e^{-\alpha}$).

Now, if $i$ is even, then the jump at $t_{i+1}$ is from state $A$ to state $B$ and has probability density $p_{i}(t_{i+1}\vert t_{i})=\alpha e^{-(t_{i+1}-t_{i})\alpha}$. Otherwise the jump is from state $B$ to state $A$ and has probability density $p_{i}(t_{i+1}\vert t_{i})=\beta e^{-(t_{i+1}-t_{i})\beta}$. Plugging into $(2)$ and simplifying gives $$p(\vec{t}\vert A)=e^{-[t_{A}(\vec{t})\alpha+(1-t_{A}(\vec{t}))\beta]}\alpha^{x/2}\beta^{x/2}\tag{3},$$ where $t_A(\vec{t})$ indicates the amount of time that trajectory $\vec{t}$ spends in state $A$.

To get your answer, we integrate over the set of all ordered sequences $\vec{t}$ that obey $(1)$ and spend $t_A$ time in state $A$, which we indicate as $\Omega$. This gives: \begin{align}P(A,x,t_{A}\vert A)=\int_{\Omega}p(\vec{t}\vert A)d\vec{t}=e^{-[t_{A}\alpha+(1-t_{A})\beta]}\alpha^{x/2}\beta^{x/2}\int_{\Omega} d\vec{t}.\tag{4}\end{align}

It remains to calculate the volume of $\Omega$. Let us indicate this set as $$\Omega=\{t\in\mathbb{R}_{+}^{x}:0\le t_{1}\le\dots t_{x}\le1,\sum_{i=1}^{x/2}t_{2i}-t_{2i-1}=1-t_{A}\}$$ Note that $\Omega$ has the same volume as the following subset of the unit $x$-simplex: $$\Omega'=\{g\in\mathbb{R}_{+}^{x+1}:\sum_{i=1}^{x+1} g_{i}=1,\sum_{i=1}^{x/2}g_{2i}=1-t_{A}\},$$ which follows from the simple transformation $g_{i}=t_{i}-t_{i-1}$ for $i=1..x+1$ (where $t_0=0$ and $t_{x+1}=1$). Then, by rearranging the odd and even coordinates, it is easy to see that $\Omega'$ is itself a cross product of two scaled unit simplices: $$\Omega'=\{g\in\mathbb{R}_{+}^{x/2+1}:\sum_{i=1}^{x/2+1}g_{i}=t_{A}\}\times\{g\in\mathbb{R}_{+}^{x/2}:\sum_{i=1}^{x/2} g_{i}=1-t_{A}\}$$ The volume of $\Omega'$ is the product of the volumes of these simplices, $$\mathrm{Vol}(\Omega') =\mathrm{Vol}(\Omega) = \frac{t_{A}^{x/2}}{(x/2)!}\frac{(1-t_{A})^{x/2-1}}{(x/2-1)!}$$ Combined with $(4)$ this gives \begin{align}P(A,x,t_{A}\vert A)=e^{-[t_{A}\alpha+(1-t_{A})\beta]}\alpha^{x/2}\beta^{x/2}\frac{t_{A}^{x/2}}{(x/2)!}\frac{(1-t_{A})^{x/2-1}}{(x/2-1)!}.\end{align}