A simple problem in markov chains - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T05:44:54Z http://mathoverflow.net/feeds/question/44838 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/44838/a-simple-problem-in-markov-chains A simple problem in markov chains The man in the box 2010-11-04T15:51:37Z 2010-12-04T10:28:35Z <p>I'm trying to understand a 1954 paper of Kubo intitled "Note on the stochastic theory of resonance absorption". The specific problem can be stated mathematically as follows: let $X(t)$ be a random process taking $n$ positive real values ${\omega_1,...,\omega_n}$. Suppose that $X(t)$ is Markov and its probablility transitions $P_{ij}(t) := P(X(t)= \omega_j | X(0) = \omega_i)$ satisfy</p> <p>$P'_{jk}(t)= -c_kP_{jk}(t)+\sum_m P_{jm}(t)c_m P_{mk}$,</p> <p>where $p_{ii}=0$ and $\sum_j p_{ij} = 0$.</p> <p>We want to find the expectation value of $M(t):=\exp(i\int_0^t X(t')dt')$ in terms of parameters $\omega_i$, $c_i$ and $p_{ij}$.</p> <p>Kubo's strategy seems to condition on $X(0) = \omega_i \wedge X(t) = \omega_j$ so he can find a link to the $P_{ij}(t)$, but I don't understand this too much... specifically he introduce a funtion $Q_{ij}(t)$ which is the average of $M(t)$ on the condition that the process is in $\omega_i$ at time $t=0$ and is found in the state $\omega_k$ at time $t$. He concludes that</p> <p>$Q'_{jk}(t) = (i\omega_k-c_k)Q_{jk} (t) + \sum_m Q _{jm}(t)c_m p_{mk}$</p> <p>I can't figure how to get to this conclusion.</p> http://mathoverflow.net/questions/44838/a-simple-problem-in-markov-chains/47607#47607 Answer by Didier Piau for A simple problem in markov chains Didier Piau 2010-11-28T19:20:40Z 2010-12-04T10:28:35Z <p>Indeed, these formulas are standard. Their derivation in a slighly more general setting than yours is as follows. </p> <p>For every $t\ge0$, let $$M_t=\displaystyle\exp\left(\int_0^tv(X_s)\mathrm{d}s\right),$$ for a given function $v$ defined on the state space of the process $(X_t)$. (In your setting, every $X_t$ is real valued and $v(x)=\mathrm{i}x$ for every $x$ but these details are irrelevant.) For every states $x$ and $y$, let $$A_t(y)=[X_t=y],\qquad Q_t(x,y)=E(M_t1_{A_t(y)}\vert X_0=x).$$ Let $Q_t$ denote the associated square matrix (indexed by the state space, possibly infinite). For instance, $Q_0$ is the identity matrix. Note also that in the expression of $Q_t(x,y)$, $[X_0=x]$ appears as a <b>conditioning</b> while $A_t(y)=[X_t=y]$ is the event to which the expectation is <b>restricted</b> and that these are different operations hence your interpretation of Kubo's method should be rephrased.</p> <p>The dynamics of $(Q_t)$ is driven by a linear differential equation $Q'_t=GQ_t$, where $G$ is a deformation of the infinitesimal generator of the process $(X_t)$. To identify $G$, one can compute $Q_{t+s}$ at the order $s$, for $s > 0$, when $s$ is small. </p> <p>To do so, call $r(x,y)$ the transition rate of $(X_t)$ from $x$ to $y\ne x$, and $c(x)$ the sum over $y\ne x$ of $r(x,y)$. (In Kubo's setting as reproduced in your post, $c(x)$ is your $c_x$ and $r(x,y)$ is your $c_xp_{xy}$. By the way, the sum over $y\ne x$ of your $p_{xy}$ should be $1$ instead of $0$ and you should make up your mind between the notations $p_{xy}$ and $P_{xy}$.)</p> <p>Then, conditioning on $[X_0=x]$, one can decompose the expectation which defines $Q_{t+s}(x,y)$ along the values of $X_s$. This decomposition goes as follows. For every $z\ne x$, $X_s=z$ with probability $r(x,z)s+o(s)$, and $X_s=x$ with probability $1-c(x)s+o(s)$. Furthermore, for every $z\ne x$, $M_{t+s}=(1+o(1))M_t$ on $[X_0=x,X_s=z]$. And on $[X_0=X_s=x]$, the probability of a double transition in the time interval $[0,s]$ is $o(s)$, hence $M_{t+s}=(1+v(x)s+o(s))M_{t+s}/M_s$ where $M_{t+s}/M_s$ is distributed like $M_t$ conditional on $[X_0=x]$.</p> <p>All this leads to $$Q_{t+s}(x,y)=Q_t(x,y)(1+v(x)s)(1-c(x)s)+\sum_zQ_t(z,y)r(x,z)s+o(s).$$ When $s\to0$, one gets $$Q'_t(x,y)=(v(x)-c(x))Q_t(x,y)+\sum_zr(x,z)Q_t(z,y).$$ In other words, $G(x,x)=v(x)-c(x)$ for every $x$ and $G(x,y)=r(x,y)$ for every $y\ne x$. These are the equations in Kubo's paper. </p>