Let $P,Q,R$ be probability measures on the real line. If these are discrete, we can show \begin{equation} D_{\mathrm{KL}}(P\ast R\,\Vert\,Q\ast R)\le D_{\mathrm{KL}}(P\,\Vert\,Q) \end{equation} by using the log sum inequality, where $D_{\mathrm{KL}}(P\,\Vert\,Q)=\int \log\left(\frac{dP}{dQ}\right) dP$ is KL divergence and $\ast$ is convolution. Is the above inequality true when $P,Q,R$ are not necessarily discrete?
1 Answer
$\begingroup$
$\endgroup$
The KL divergence cannot increase after passing both distributions through the same Markov kernel (in your case, convolution with $R$). This is an immediate consequence of the data processing inequality: https://en.wikipedia.org/wiki/Data_processing_inequality