1
$\begingroup$

Let $P,Q,R$ be probability measures on the real line. If these are discrete, we can show \begin{equation} D_{\mathrm{KL}}(P\ast R\,\Vert\,Q\ast R)\le D_{\mathrm{KL}}(P\,\Vert\,Q) \end{equation} by using the log sum inequality, where $D_{\mathrm{KL}}(P\,\Vert\,Q)=\int \log\left(\frac{dP}{dQ}\right) dP$ is KL divergence and $\ast$ is convolution. Is the above inequality true when $P,Q,R$ are not necessarily discrete?

$\endgroup$
1
  • $\begingroup$ I am specially interested in a case where R is normal distribution. $\endgroup$
    – selami
    Commented Feb 12, 2019 at 7:05

1 Answer 1

1
$\begingroup$

The KL divergence cannot increase after passing both distributions through the same Markov kernel (in your case, convolution with $R$). This is an immediate consequence of the data processing inequality: https://en.wikipedia.org/wiki/Data_processing_inequality

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .