2
$\begingroup$

Consider a continuous-time, irreducible Markov chain $X_t$ on a finite state space $E$. Assume the jump rates are $R(x,y)$ for $x,y\in E$, the generator is $L$, i.e for any function $f$ on E, $$Lf(x)=\sum_{y}R(x,y)[f(y)-f(x)].$$ Assume also the chain is reversible w.r.t. the measure $\pi$. Suppose the chain starts from an initial distribution $\mu$. Denote by $\mu P_t$ the distribution of chain at time $t$. Define the density $f_t=\frac{\mu P_t}{\pi}$. The corresponding entropy is given by $H(f_t)=\sum_{x\in E} f_t(x)\log f_t(x)\pi(x)$.

It is well know that $H(f_t)$ is decreasing in time $t$. More precisely, we have $$H(f_0)-H(f_t)=\int_0^t-\sum_xf_s(x)(L\log f_s)(x)\pi(x)ds.$$ Lets call the term inside the integral at the right hand side of the equation $E(f_t)$. Note that $E(f_t)$ is positive because it is bounded from below by $$D(\sqrt{f_s})=\sum_{x,y}R(x,y)[\sqrt{f_s(y)}-\sqrt{f_s(x)}]^2\pi(x),$$ by the inequality $a[\log b-\log a]\leq 2\sqrt{a}[\sqrt{b}-\sqrt{a}]$ for any $a,b\geq 0$.

$f_t$ eventually goes to the constant $1$, so $D(\sqrt{f_t})$ goes to $0$. My question is: is the true that $D(\sqrt{f_t})$ is decreasing along the time? What about $E(f_t)$?

$\endgroup$

1 Answer 1

1
$\begingroup$

I don't believe the answer is known in general.

However, if you allow $R(x,y)$ to not be reversible w.r.t to $\pi$ (so that $\pi$ is the "non-equilibrium" stationary distribution of $R$, in physics language), then the answer is no: neither $D(\sqrt{f_t})$ nor $E(f_t)$ (which in physics is called the entropy production rate) has to be decreasing along time. To see a counterexample for $E(f_t)$, see Eq.3-5 in

  • Polettini and Esposito, "Nonconvexity of the relative entropy for Markov dynamics: A Fisher information approach", Physical Review E, 2013.

It can be seen numerically that, for their counterexample, $D(\sqrt{f_t})$ is also not monotonically decreasing.

In that paper they do prove that $E(f_t)$ is going to be monotonically decreasing if (1) the Markov chain is reversible, and (2) the system is sufficiently close to equilibrium.

$\endgroup$
1
  • $\begingroup$ Thanks for the reference. It's nice to know the answer is negative for the nonreversible case. $\endgroup$
    – Tiago
    Commented Apr 4, 2021 at 13:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .