Assume $P_1,P_2,P_3$ different to each other pmfs. We would like to find an upper bound for $D_{\text{KL}}(P_1\ast P_3||P_2 \ast P_3)$, where $D_{KL}$ is the Kullback-Leibler divergence and $*$ is convolution. By using the log sum inequality we can get $D_{\text{KL}}(P_1\ast P_3||P_2 \ast P_3) \leq D_{\text{KL}}(P_1||P_2 )$. Is there any way to get a tighter bound?

  • 3
    $\begingroup$ Where does this question come from? $\endgroup$
    – Igor Rivin
    Jun 18, 2017 at 1:48
  • $\begingroup$ it looks like it is a strict inequality ($P_1 \neq P_2 $), however, I was wondering if $P_3$ could participate to the right part. $\endgroup$
    – Cauchy
    Jun 18, 2017 at 5:17
  • 1
    $\begingroup$ Since this is a duplicate of math.stackexchange.com/questions/2326512/… you should at least mention that you are asking in more than one place, to prevent unnecessary duplication of effort. That is a basic curtesy to people who are working for free. $\endgroup$
    – S. Carnahan
    Jun 18, 2017 at 7:24
  • $\begingroup$ sorry I didn't know that $\endgroup$
    – Cauchy
    Jun 18, 2017 at 7:59
  • 1
    $\begingroup$ Isn't this essentially the data processing inequality? $\endgroup$
    – Ant
    Oct 18, 2017 at 13:03


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.