Return to Question

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

$f_x(x)\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2})$

$\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{||x||_2^2x^2_i}{||x||_2^2v_i^2})$

Now let $y_i = x_i/||x||_2$ and $u=||x||_2$. Making the transformation gives

$f_{u,y}(u,y) \propto (\prod_p v_i)^{-1} u^{p-1}\exp(-\frac{u^2}{2}\sum_p \frac{y^2_i}{v_i^2})$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2 dimensional case if $V=diag(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

\begin{align} f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2}\right) \\[10pt] & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{\|x\|_2^2x^2_i}{\|x\|_2^2v_i^2}\right) \end{align}

Now let $y_i = x_i/\|x\|_2$ and $u=\|x\|_2$. Making the transformation gives

$$f_{u,y}(u,y) \propto \left(\prod_p v_i\right)^{-1} u^{p-1}\exp\left(-\frac{u^2}{2} \sum_p \frac{y^2_i}{v_i^2}\right)$$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2-dimensional case if $V=\operatorname{diag}(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

$f_x(x)\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2})$

$\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{||x||^2x^2_i}{||x||^2v_i^2})$

Now let $y_i = x_i/||x||$ and $u=||x||$. Making the transformation gives

$f_{u,y}(u,y) \propto (\prod_p v_i)^{-1} u^{p-1}\exp(-\frac{u^2}{2}\sum_p \frac{y^2_i}{v_i^2})$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2 dimensional case if $V=diag(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

$f_x(x)\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2})$

$\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{||x||_2^2x^2_i}{||x||_2^2v_i^2})$

Now let $y_i = x_i/||x||_2$ and $u=||x||_2$. Making the transformation gives

$f_{u,y}(u,y) \propto (\prod_p v_i)^{-1} u^{p-1}\exp(-\frac{u^2}{2}\sum_p \frac{y^2_i}{v_i^2})$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2 dimensional case if $V=diag(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

$f_x(x)\propto (\prod_p v_i) \exp(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2})$

$\propto (\prod_p v_i) \exp(-\frac{1}{2}\sum_p \frac{||x||^2x^2_i}{||x||^2v_i^2})$

Now let $y_i = x_i/||x||$ and $u=||x||$. Making the transformation gives

$f_{u,y}(u,y) \propto (\prod_p v_i) u^{p-1}\exp(-\frac{u^2}{2}\sum_p \frac{y^2_i}{v_i^2})$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2 dimensional case if $V=diag(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

$f_x(x)\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2})$

$\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{||x||^2x^2_i}{||x||^2v_i^2})$

Now let $y_i = x_i/||x||$ and $u=||x||$. Making the transformation gives

$f_{u,y}(u,y) \propto (\prod_p v_i)^{-1} u^{p-1}\exp(-\frac{u^2}{2}\sum_p \frac{y^2_i}{v_i^2})$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2 dimensional case if $V=diag(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.