MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider a finite state, irreducible Markov chain with a rate matrix $Q$ and a stationary distribution $\pi$. Suppose the chain starts with the initial distribution $p$ at time $0$, then at time $t$ the distribution is given by $$p_t = p*e^{Qt}$$ The relative entropy of $p_t$ with respect to the $\pi$ (also known as the Kullback-Leibler divergence $D(p_t || \pi)$) is given by $$D(p_t || \pi) = \sum_i p_t(i) \log \left(\frac{p_t(i)}{\pi(i)}\right)$$ where $i$ ranges over all the states.

It seems well known that $D(p_t || \pi)$ monotonically decreases as $t$ increases. (For instance, section $2.9$ of Cover and Thomas's book here https://web.cse.msu.edu/~cse842/Papers/CoverThomas-Ch2.pdf).

My question is:

Is $D(p_t || \pi)$ a convex function of $t$?

I simulated for many random matrices using Matlab and did not find a counter example. Any references would be appreciated!

share|cite|improve this question
up vote 4 down vote accepted

Turns out $D(p_t||\pi)$ need not always be convex. The following paper demonstrates a counter-example in section $4.2$- http://arxiv.org/abs/0712.2578

share|cite|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.