Assume $P_1,P_2,P_3$ different to each other pmfs. We would like to find an upper bound for $D_{\text{KL}}(P_1\ast P_3||P_2 \ast P_3)$, where $D_{KL}$ is the Kullback-Leibler divergence and $*$ is convolution. By using the log sum inequality we can get $D_{\text{KL}}(P_1\ast P_3||P_2 \ast P_3) \leq D_{\text{KL}}(P_1||P_2 )$. Is there any way to get a tighter bound?
9
-
3$\begingroup$ Where does this question come from? $\endgroup$– Igor RivinCommented Jun 18, 2017 at 1:48
-
$\begingroup$ it looks like it is a strict inequality ($P_1 \neq P_2 $), however, I was wondering if $P_3$ could participate to the right part. $\endgroup$– CauchyCommented Jun 18, 2017 at 5:17
-
1$\begingroup$ Since this is a duplicate of math.stackexchange.com/questions/2326512/… you should at least mention that you are asking in more than one place, to prevent unnecessary duplication of effort. That is a basic curtesy to people who are working for free. $\endgroup$– S. Carnahan ♦Commented Jun 18, 2017 at 7:24
-
$\begingroup$ sorry I didn't know that $\endgroup$– CauchyCommented Jun 18, 2017 at 7:59
-
1$\begingroup$ Isn't this essentially the data processing inequality? $\endgroup$– AntCommented Oct 18, 2017 at 13:03
|
Show 4 more comments