1
$\begingroup$

Consider a two-state Markov process in continuous time, with states labelled $A$ and $B$. The transition rates for going from state $A$ to $B$, and state $B$ to $A$ are $\alpha$ and $\beta$ respectively. We are interested in the quantity $p_t(A,x,t_A|A)$, which denotes the probability that, at time $t$, the Markov process is in state $A$, has made a total of $x$ jumps, and has spent time equal to $t_A$ in state $A$, given that it started from state $A$.

I was thinking of two approaches (and both are likely to be equivalent). But I seem to be running into hurdles in both of them, and would be grateful for some help.

My first approach was to write a set of coupled recurrence relations in the following way: \begin{align} p_t(A,x,t_A|A) &= \int_0^t e^{-\alpha t'}\cdot \alpha \cdot p_{t-t'}(A,x-1,t_A-t'|B)\cdot dt'\\ p_t(A,x,t_A|B) &= \int_0^t e^{-\beta t'}\cdot \beta \cdot p_{t-t'}(A,x-1,t_A|A)\cdot dt' \end{align} The basic idea behind the above equations is that the first jump takes place at time $t'$, and thus, the rest of the $x-1$ jumps have to be made in the remaining $t-t'$ amount of time. Similarly, depending on what the initial state was, the time needed to be spent in state $A$ in the remaining $t-t'$ time is stated in the argument (e.g. if the first $t'$ amount of time was spent in state $A$, then in the remaining amount of time, only $t_A-t'$ amount of time needs to be spent there). There is a minor issue here that the above equation does not necessarily restrict the possibility of $t'>t_A$, which is unphysical. But that can be taken care of by changing the upper limit of the integral in the first equation to $t_A$.

Naively, one would expect that taking a Laplace transform of the two equations with respect to $t$ should simplify matters a lot. For the second equation, it indeed does, as the equation is a simple convolution in $t$. However, the first is not (please correct me if I am wrong) and thus, naive Laplace transform with respect to $t$ might not be helpful. Is there any other approach which makes the problem easier?

My second approach was to derive the differential equation satisfied by the quantity $p_t(A,x,t_A|A)$, by working in discrete time-steps of $\Delta t$, and finally taking the limit $\Delta t \to 0$:

\begin{align} p_{t+\Delta t}(A,x,t_A|A) = p_t(A,x,t_A-\Delta t|A)\cdot (1-\alpha \Delta t) + p_t(B,x-1,t_A-\Delta t|A)\cdot \beta \Delta t \end{align} which upon rearranging gives us \begin{align} \frac{p_{t+\Delta t}(A,x,t_A|A) - p_{t}(A,x,t_A- \Delta t|A)}{\Delta t} = -\alpha\cdot p_t(A,x,t_A-\Delta t|A) + \beta \cdot p_t(B,x-1,t_A - \Delta t|A). \end{align} In the limit of $\Delta t \to 0$, we get: $$\frac{\partial{p_{t}(A,x,t_A|A)}}{\partial{t}}+\frac{\partial{p_{t}(A,x,t_A|A)}}{\partial{t_A}} = -\alpha\cdot p_t(A,x,t_A|A) + \beta \cdot p_t(B,x-1,t_A|A). $$

Similarly, one could write equations for $p_t(B,x,t_A|A)$. Here (again naively), it seems like one could take a Laplace transform with respect to $t$ and $t_A$, and define a generating function for $x$ to simplify the problem. However, one thing that scares me is that $t_A$ cannot take any value. It must always take values less than $t$. Thus, independently taking Laplace transforms with respect to $t$ and $t_A$ seems shady.

Is there a way to overcome one (or both) of the hurdles?

$\endgroup$
1
  • 1
    $\begingroup$ Did you try to calculate these functions using your recurrence for $x=1,2,3$? $\endgroup$ Oct 9, 2021 at 21:37

1 Answer 1

1
+100
$\begingroup$

Without loss of generality, let the final time to be $t=1$ (if it is not, we can make it so by rescaling time as $\alpha'=t\alpha$ and $\beta'=t\alpha$).

Then, consider a single trajectory of the process that starts and ends on state $A$ and makes $x$ jumps ($x$ has to be even). We can represent any such trajectory in terms of the jump times, $$\vec{t}=(t_{0}=0\le t_{1}\le\dots\le t_{x}\le1) \tag{1}$$ The conditional probability of the trajectory is given by the product of the jump probabilities, multiplied by the probability of not leaving state $A$ after the last jump: $$p(\vec{t}\vert A)=e^{-\alpha(1-t_{x})}\prod_{i=0}^{x-1}p_{i}(t_{i+1}\vert t_{i}).\tag{2}$$ We will assume below that $x\ge 2$ (otherwise $t_A=1$ and $p(A,x,t_A|A)=e^{-\alpha}$).

Now, if $i$ is even, then the jump at $t_{i+1}$ is from state $A$ to state $B$ and has probability density $p_{i}(t_{i+1}\vert t_{i})=\alpha e^{-(t_{i+1}-t_{i})\alpha}$. Otherwise the jump is from state $B$ to state $A$ and has probability density $p_{i}(t_{i+1}\vert t_{i})=\beta e^{-(t_{i+1}-t_{i})\beta}$. Plugging into $(2)$ and simplifying gives $$p(\vec{t}\vert A)=e^{-[t_{A}(\vec{t})\alpha+(1-t_{A}(\vec{t}))\beta]}\alpha^{x/2}\beta^{x/2}\tag{3},$$ where $t_A(\vec{t})$ indicates the amount of time that trajectory $\vec{t}$ spends in state $A$.

To get your answer, we integrate over the set of all ordered sequences $\vec{t}$ that obey $(1)$ and spend $t_A$ time in state $A$, which we indicate as $\Omega$. This gives: \begin{align}P(A,x,t_{A}\vert A)=\int_{\Omega}p(\vec{t}\vert A)d\vec{t}=e^{-[t_{A}\alpha+(1-t_{A})\beta]}\alpha^{x/2}\beta^{x/2}\int_{\Omega} d\vec{t}.\tag{4}\end{align}

It remains to calculate the volume of $\Omega$. Let us indicate this set as $$\Omega=\{t\in\mathbb{R}_{+}^{x}:0\le t_{1}\le\dots t_{x}\le1,\sum_{i=1}^{x/2}t_{2i}-t_{2i-1}=1-t_{A}\}$$ Note that $\Omega$ has the same volume as the following subset of the unit $x$-simplex: $$\Omega'=\{g\in\mathbb{R}_{+}^{x+1}:\sum_{i=1}^{x+1} g_{i}=1,\sum_{i=1}^{x/2}g_{2i}=1-t_{A}\},$$ which follows from the simple transformation $g_{i}=t_{i}-t_{i-1}$ for $i=1..x+1$ (where $t_0=0$ and $t_{x+1}=1$). Then, by rearranging the odd and even coordinates, it is easy to see that $\Omega'$ is itself a cross product of two scaled unit simplices: $$\Omega'=\{g\in\mathbb{R}_{+}^{x/2+1}:\sum_{i=1}^{x/2+1}g_{i}=t_{A}\}\times\{g\in\mathbb{R}_{+}^{x/2}:\sum_{i=1}^{x/2} g_{i}=1-t_{A}\}$$ The volume of $\Omega'$ is the product of the volumes of these simplices, $$\mathrm{Vol}(\Omega') =\mathrm{Vol}(\Omega) = \frac{t_{A}^{x/2}}{(x/2)!}\frac{(1-t_{A})^{x/2-1}}{(x/2-1)!}$$ Combined with $(4)$ this gives \begin{align}P(A,x,t_{A}\vert A)=e^{-[t_{A}\alpha+(1-t_{A})\beta]}\alpha^{x/2}\beta^{x/2}\frac{t_{A}^{x/2}}{(x/2)!}\frac{(1-t_{A})^{x/2-1}}{(x/2-1)!}.\end{align}

$\endgroup$
8
  • $\begingroup$ Thanks a lot for the answer -- I was fairly convinced by your logic. However, I am slightly concerned about the final two steps. When you integrate over all $\Omega$ (all admissible combinations of $t_1, t_2, \dots, t_x$), aren't you integrating over all $t_A$ also? So perhaps $\Omega$ should be restricted in such a way, that $t_A$ remains a constant. I have been trying a different approach to the problem, and will post my solution shortly. $\endgroup$ Oct 21, 2021 at 9:58
  • 1
    $\begingroup$ @StatisticalMechanic I think you are absolutely right, my original answer was not correct because I integrated over $t_A$ -- apologies. However, I think there is a simple fix, which is to represent the set of "constrained" trajectories (where each trajectory is required to spend time $t_A$ in state $A$) is written as a cross product of two simplices. I have updated my answer. $\endgroup$
    – Artemy
    Oct 21, 2021 at 17:24
  • $\begingroup$ This is fantastic, thanks! Just one more thing -- we do not want to compute the volume of these simplices, right? Rather, we want their surface area. For example, if we take $x=3$, then we have $g_1,g_2,g_3$ and $g_4$ such that $g_1+g_3=t_A$ and $g_2+g_4=1-t_A$. Now, we are not interested in the area of the right angled triangle formed by the $g_1$ and $g_3$ axes, and the line $g_1+g_3=t_A$. Instead, we want the length of the line with endpoints $(0,t_A)$ and $(t_A,0)$, because that defines the number of ways $g_1$ and $g_3$ can be distributed. Does that make sense? $\endgroup$ Oct 23, 2021 at 10:35
  • $\begingroup$ Perhaps we will need to use this result: math.stackexchange.com/questions/2996038/… $\endgroup$ Oct 23, 2021 at 10:39
  • 1
    $\begingroup$ @StatisticalMechanic I don't think your counterexample works, as $x$ has to be even for the trajectory to end on state A. Instead, let us take $x=2$ as an example. Then, $g_1+g_3=t_A$ and $g_2=1-t_A$, and the volume of $\Omega'$ is given by the volume of a simplex with two vertices (i.e., a line) and a simplex with one vertex (i.e., a point), as expected. $\endgroup$
    – Artemy
    Oct 25, 2021 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.