Given a random variable $X$ that is uniformly distributed on $[-b,b]$ and $Y=g(X)$ with $$g(x) = \begin{cases} 0, ~~~ x\in [-c,c] \\ x, ~~~ \text{else}\end{cases}$$

Now I want to compute the information dimension $d(X), d(Y)$ and the conditional information dimension $d(X|Y)$ and show that $d(X) = d(X|Y) + d(Y)$ in this case.

The information dimension is defined as $$ d(X) = \lim_{m\rightarrow \infty} \frac{H(\hat{X}^{(m)})}{m} $$ with $$ \hat{X}^{(m)} := \frac{\lfloor2^m X \rfloor}{2^m} $$ the quantization of $X$.

For a discrete distribution, $d(X) = 0$, and for a continuous one-dimensional distribution, $d(X) = 1$. For a mixed distribution with discrete and continuous components of the form $P_X = d P_X^{(ac)} + (1-d) P_X^{(d)}$, the information dimension is $d(X)=d$.

Now I know, that the random variable X has a continuous component $\Rightarrow d(X) = 1$. The distribution $P_Y$ is a discrete-continuous mixture: $$ P_Y = \begin{cases} \frac{c}{b}, ~~~Y=0\\ \frac{1}{2b},~~~Y \in [-b,-c] \cap [c,b]\\ 0,~~~\text{else} \end{cases}$$ Therefore, $d(Y)=\frac{b-c}{b}$.

Now my question is the following: how do I compute the conditional information dimension? $$d(X|Y) = \lim_{m \rightarrow \infty} \frac{H(\hat{X}^{(m)}|Y)}{m} = \int_\mathcal{Y} d(X|Y=y)dP_Y(y) = \mathbb{E}_{Y\sim P_Y}(d(X|Y=Y))$$


From the context, it appears that $b\in(0,\infty)$ and $c\in[0,\infty)$ (and, likely, $c\le b$). Anyway, let $c_1:=\min(b,c)$. Note that

  • With probability $1$, either $Y=0$ or $c_1<|Y|\le b$;
  • The conditional distribution of $X$ given $Y=0$ is the uniform distribution on the interval $[-c_1,c_1]$ and hence $d(X|Y=0)=1$.
  • The conditional distribution of $X$ given $Y=y$ with $c_1<|y|\le b$ is the Dirac distribution supported at point $y$ and hence $d(X|Y=y)=0$.

So, $$d(X|Y)=\int_{\mathbb R}d(X|y)P(Y\in dy)=P(Y=0)=\frac{c_1}b=\frac{\min(b,c)}b.$$

  • $\begingroup$ oh makes sense, thank you a lot! $\endgroup$
    – Phobos
    Jul 31 '20 at 5:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.