# Dependence between direction and magnitude of multivariate normal random vector

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

$f_x(x)\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2})$

$\propto (\prod_p v_i)^{-1} \exp(-\frac{1}{2}\sum_p \frac{||x||_2^2x^2_i}{||x||_2^2v_i^2})$

Now let $y_i = x_i/||x||_2$ and $u=||x||_2$. Making the transformation gives

$f_{u,y}(u,y) \propto (\prod_p v_i)^{-1} u^{p-1}\exp(-\frac{u^2}{2}\sum_p \frac{y^2_i}{v_i^2})$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2 dimensional case if $V=diag(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

-
The reasoning looks correct. I think you can re-parametrize by keeping the direction $y$ the same, and just dividing $x$ by the standard deviation (point-wise, that is, $x_i' = x_i/v_i$) in $u$, that is, defining $u$ as $u = || (x_1 / v_1, .., x_p/v_p) ||$. B.t.w. isn't $f_x(x)$ proportional to $1/\prod_i v_i$ (instead of $\prod_p v_i$)? – Or Zuk Apr 14 '11 at 19:06
Of course, fixed. Thanks. – JMS Apr 14 '11 at 20:22

Your reasoning looks right, although I'm not that familiar with the exact notation you're using, except that the $v_i$ should be in the denominator, not the numerator.

In the second case the answer is yes. In general, say you have any norm $\| \cdot \|$ on $\mathbb{R}^p$. There is a measure $\mu$ on the boundary of the unit ball $B$ of $\| \cdot \|$, called the cone measure, with the property that there is the following version of integration in spherical coordinates: $$\int_{\mathbb{R}^p} f(x) \ dx = \int_0^\infty u^{p-1} \int_{\partial B} f(uy) \ d\mu(y) \ du$$ for any integrable function $f$.

Now in your case your density can be written in the form $f(x) = F(\| x \|)$, where $\| x \| = \sqrt{\sum (x_i/v_i)^2}$. This means that a random vector $X \sim N(0,V)$ has the property that $\| X \|$ and $X/\|X\|$ are independent, and the latter is distributed according to the cone measure on the surface of the ellipsoid $\{ x : \| x \|_V \le 1\}$.

-
Yeah, the $v_i$'s should have been inverted. Fixed now. As you correctly inferred, I was using $||x|| = \sqrt(x'x)$, I made them into $||x||_2$ which is hopefully clearer. Thanks – JMS Apr 14 '11 at 20:44

Every nonnegative-definite symmetric real matrix is the matrix of covariances of the components of some random vector---that follows from the finite-dimensional spectral theorem. In Feller's terminology, the variance of a random vector $X$ is $E((X-\mu)(X-\mu)^T)$, where $\mu=E(X)$, so it is just the matrix of covariances. Now suppose $X$ is a random vector that is normally distributed with expected value $0$ and variance $M$, where $M$ is some positive-definite symmetric matrix. For now I'll assume $M$ is nonsingular. It is well-known that $M$ must have a positive-definite symmetric square root $M^{1/2}$. Then $M^{-1/2}X$ is normally distributed and its variance is the identity matrix. So as "independent quantities analogous to the direction and magnitude" of $X$ you could use the direction and magnitude of $M^{-1/2}X$. "Analogous to" is maybe a bit vague, so I don't know if that's the sort of thing you had in mind.

-
Vague by intention :) I was sort of fishing. Initially I had taken $V$ diagonal (wlog up to a rotation of $x$). Here we arrive at the same place as @Mark Meckes generalized to any $V$. That is, taking $||x||_V^2 = x'V^{-1}x$ then $||x||_V$ and $x/||x||_V$ are independent when $x\sim N(0, V)$. Should probably dust off my linear models text! – JMS Apr 16 '11 at 1:16