MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

For $0\leq r\leq 1$ and $0 < p < 1$, define the Kullback-Leibler divergence between the Bernoulli(r) and Bernoulli(p) distributions as: $D(r||p) := r\log\frac{r}{p} + \bar{r}\log\frac{\bar{r}}{\bar{p}}$ where $\bar{r}:=1-r, \bar{p}:=1-p.$

Fix $0<\theta_0,\theta_1<1$ and $p.$

Consider the function

$$\eta(r) := \frac{D(r\theta_1+\bar{r}\theta_0||p\theta_1+\bar{p}\theta_0)}{D(r||p)}.$$

This function can be made continuous at $r=p$ if we define $\eta(p)$ suitably.

Plots for different values of $p,\theta_0,\theta_1$ show that $\eta(r)$ is concave in $r$. The second derivative of $\eta(\cdot)$ seems quite involved though. Perhaps I am ignorant of some sophisticated techniques that could be useful here. I would be grateful for a proof that $\eta(\cdot)$ is concave. Or any ideas that could help in proving such a thing.

share|cite|improve this question
+1, nice question! so far only the case $\theta_1=\theta_0$ is easy because for that $\eta(r)=0$! For other choices, it seems to be a nontrivial problem. – Suvrit Mar 29 '13 at 17:12
This function is not concave in general for some parameters. It was discovered this is the case when I plotted it for various parameters. I don't recall now what those were, but the answer to the question is negative. – Hedonist Apr 5 at 16:22

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.