Timeline for On the difference of conditional differential entropy of two correlated random variables

7 events

when toggle format	what		by	license	comment
Aug 3, 2019 at 21:09	comment	added	stochasticboy321		Very concretely - consider the following: let $G,Z$ be two independent standard Gaussians, and $S = G/K + Z$ for a given constant $K \gg1$. Then $h(S\|G) = h(Z)$ but $h(G\|S) = h(-KZ)$, and the latter is much larger (differential entropy in not invariant to scaling). Something similar should hold for discrete $Z$, although you will have to work harder to show it.
Aug 3, 2019 at 21:09	comment	added	stochasticboy321		This is not true in general. Further, your proof strategy is off - you need to at least use some information about the spread of the $\mu_g$s. For example, suppose $\mu_g$ is identical for all $g,$ and $\sigma^2$ is small, then your expression is obviously negative. Now, by continuity of differential entropy, if I perturb the $\mu_g$ a little, so that they are all distinct, but still very close, the same should hold. How well spread the $\mu_g$ are has to enter your considerations, possibly as a condition for your desired inequality.
Jul 27, 2019 at 15:53	history	edited	Xi Chen	CC BY-SA 4.0	added 2049 characters in body
Jul 26, 2019 at 14:07	history	edited	Xi Chen	CC BY-SA 4.0	added 256 characters in body
Jul 25, 2019 at 0:25	history	edited	Xi Chen	CC BY-SA 4.0	edited body
Jul 24, 2019 at 23:25	review	First posts
Jul 25, 2019 at 4:16
Jul 24, 2019 at 23:22	history	asked	Xi Chen	CC BY-SA 4.0