Given a random variable $X$ that is uniformly distributed on $[-b,b]$ and $Y=g(X)$ with $$g(x) = \begin{cases} 0, ~~~ x\in [-c,c] \\ x, ~~~ \text{else}\end{cases}$$

Now I want to compute the information dimension $d(X), d(Y)$ and the conditional information dimension $d(X|Y)$ and show that $d(X) = d(X|Y) + d(Y)$ in this case.

The information dimension is defined as $$ d(X) = \lim_{m\rightarrow \infty} \frac{H(\hat{X}^{(m)})}{m} $$ with $$ \hat{X}^{(m)} := \frac{\lfloor2^m X \rfloor}{2^m} $$ the quantization of $X$.

For a discrete distribution, $d(X) = 0$, and for a continuous one-dimensional distribution, $d(X) = 1$. For a mixed distribution with discrete and continuous components of the form $P_X = d P_X^{(ac)} + (1-d) P_X^{(d)}$, the information dimension is $d(X)=d$.

Now I know, that the random variable X has a continuous component $\Rightarrow d(X) = 1$. The distribution $P_Y$ is a discrete-continuous mixture: $$ P_Y = \begin{cases} \frac{c}{b}, ~~~Y=0\\ \frac{1}{2b},~~~Y \in [-b,-c] \cap [c,b]\\ 0,~~~\text{else} \end{cases}$$ Therefore, $d(Y)=\frac{b-c}{b}$.

Now my question is the following: how do I compute the conditional information dimension? $$d(X|Y) = \lim_{m \rightarrow \infty} \frac{H(\hat{X}^{(m)}|Y)}{m} = \int_\mathcal{Y} d(X|Y=y)dP_Y(y) = \mathbb{E}_{Y\sim P_Y}(d(X|Y=Y))$$


1 Answer 1


From the context, it appears that $b\in(0,\infty)$ and $c\in[0,\infty)$ (and, likely, $c\le b$). Anyway, let $c_1:=\min(b,c)$. Note that

  • With probability $1$, either $Y=0$ or $c_1<|Y|\le b$;
  • The conditional distribution of $X$ given $Y=0$ is the uniform distribution on the interval $[-c_1,c_1]$ and hence $d(X|Y=0)=1$.
  • The conditional distribution of $X$ given $Y=y$ with $c_1<|y|\le b$ is the Dirac distribution supported at point $y$ and hence $d(X|Y=y)=0$.

So, $$d(X|Y)=\int_{\mathbb R}d(X|y)P(Y\in dy)=P(Y=0)=\frac{c_1}b=\frac{\min(b,c)}b.$$

  • $\begingroup$ oh makes sense, thank you a lot! $\endgroup$
    – Phobos
    Jul 31, 2020 at 5:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.