Conditional entropy - solve example

Question

Given a random variable $X$ that is uniformly distributed on $[-b,b]$ and $Y=g(X)$ with $$g(x) = \begin{cases} 0, ~~~ x\in [-c,c] \\ x, ~~~ \text{else}\end{cases}$$

Now I want to compute the information dimension $d(X), d(Y)$ and the conditional information dimension $d(X|Y)$ and show that $d(X) = d(X|Y) + d(Y)$ in this case.

The information dimension is defined as $$ d(X) = \lim_{m\rightarrow \infty} \frac{H(\hat{X}^{(m)})}{m} $$ with $$ \hat{X}^{(m)} := \frac{\lfloor2^m X \rfloor}{2^m} $$ the quantization of $X$.

For a discrete distribution, $d(X) = 0$, and for a continuous one-dimensional distribution, $d(X) = 1$. For a mixed distribution with discrete and continuous components of the form $P_X = d P_X^{(ac)} + (1-d) P_X^{(d)}$, the information dimension is $d(X)=d$.

Now I know, that the random variable X has a continuous component $\Rightarrow d(X) = 1$. The distribution $P_Y$ is a discrete-continuous mixture: $$ P_Y = \begin{cases} \frac{c}{b}, ~~~Y=0\\ \frac{1}{2b},~~~Y \in [-b,-c] \cap [c,b]\\ 0,~~~\text{else} \end{cases}$$ Therefore, $d(Y)=\frac{b-c}{b}$.

Now my question is the following: how do I compute the conditional information dimension? $$d(X|Y) = \lim_{m \rightarrow \infty} \frac{H(\hat{X}^{(m)}|Y)}{m} = \int_\mathcal{Y} d(X|Y=y)dP_Y(y) = \mathbb{E}_{Y\sim P_Y}(d(X|Y=Y))$$

Answer 1

From the context, it appears that $b\in(0,\infty)$ and $c\in[0,\infty)$ (and, likely, $c\le b$). Anyway, let $c_1:=\min(b,c)$. Note that

• With probability $1$, either $Y=0$ or $c_1<|Y|\le b$;
• The conditional distribution of $X$ given $Y=0$ is the uniform distribution on the interval $[-c_1,c_1]$ and hence $d(X|Y=0)=1$.
• The conditional distribution of $X$ given $Y=y$ with $c_1<|y|\le b$ is the Dirac distribution supported at point $y$ and hence $d(X|Y=y)=0$.

So, $$d(X|Y)=\int_{\mathbb R}d(X|y)P(Y\in dy)=P(Y=0)=\frac{c_1}b=\frac{\min(b,c)}b.$$