Skip to main content
1 of 6
Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229

$\newcommand\de\delta$The maps $$\mu\mapsto\sqrt{\mathrm{KL}(\mu\|\nu)}$$ and $$\nu\mapsto\sqrt{\mathrm{KL}(\mu\|\nu)}$$ are not convex in general.

Indeed, let $\mu_p:=p\de_0+(1-p)\de_1$, where $p\in(0,1)$ and $\de_a$ is the Dirac measure supported on $\{a\}$.

Then the second partial derivative with respect to $p$ of $\sqrt{KL(\mu_p,\mu_r)}$ at $(p,r)=(1/10,1/11)$ is $-7.17\ldots<0$. So, $\sqrt{KL(\mu,\mu_r)}$ is not convex in $\mu$.

Also, the second partial derivative with respect to $r$ of $\sqrt{KL(\mu_p,\mu_r)}$ at $(p,r)=(1/10,1/9)$ is $-11.50\ldots<0$. So, $\sqrt{KL(\mu_p,\nu)}$ is not convex in $\nu$.

Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229