Skip to main content
4 of 4
added 105 characters in body
Michael Hardy
  • 1
  • 12
  • 85
  • 126

Dependence between direction and magnitude of multivariate normal random vector

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

\begin{align} f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2}\right) \\[10pt] & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{\|x\|_2^2x^2_i}{\|x\|_2^2v_i^2}\right) \end{align}

Now let $y_i = x_i/\|x\|_2$ and $u=\|x\|_2$. Making the transformation gives

$$f_{u,y}(u,y) \propto \left(\prod_p v_i\right)^{-1} u^{p-1}\exp\left(-\frac{u^2}{2} \sum_p \frac{y^2_i}{v_i^2}\right)$$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2-dimensional case if $V=\operatorname{diag}(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

JMS
  • 269
  • 2
  • 11