# Dependence between direction and magnitude of multivariate normal random vector

Suppose that $$x\sim N(0, V)$$ is $$p$$ dimensional with $$V$$ diagonal having elements $$v_i^2$$. Then

\begin{align} f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2}\right) \\[10pt] & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{\|x\|_2^2x^2_i}{\|x\|_2^2v_i^2}\right) \end{align}

Now let $$y_i = x_i/\|x\|_2$$ and $$u=\|x\|_2$$. Making the transformation gives

$$f_{u,y}(u,y) \propto \left(\prod_p v_i\right)^{-1} u^{p-1}\exp\left(-\frac{u^2}{2} \sum_p \frac{y^2_i}{v_i^2}\right)$$

where $$u\in (0, \infty)$$ and $$y'y=1$$ (with $$u^{p-1}$$ coming in through the Jacobian). The density doesn't factor (unless $$V\propto I$$), so $$u$$ and $$y$$ are dependent. This is perfectly sensible to me; informally, in the 2-dimensional case if $$V=\operatorname{diag}(10000, 1)$$ then clearly if the direction is near $$(1,0)$$ the magnitude will be larger than if it were near $$(0,1)$$. Similarly, it's intuitive that the dependence disappears if $$V \propto I$$ (in which case $$y$$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $$V\not \propto I$$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $$y$$ to lie on an ellipsoid determined by $$V$$)? It seems like there should be but it's eluding me.

• The reasoning looks correct. I think you can re-parametrize by keeping the direction $y$ the same, and just dividing $x$ by the standard deviation (point-wise, that is, $x_i' = x_i/v_i$) in $u$, that is, defining $u$ as $u = || (x_1 / v_1, .., x_p/v_p) ||$. B.t.w. isn't $f_x(x)$ proportional to $1/\prod_i v_i$ (instead of $\prod_p v_i$)? Commented Apr 14, 2011 at 19:06

Your reasoning looks right, although I'm not that familiar with the exact notation you're using, except that the $v_i$ should be in the denominator, not the numerator.

In the second case the answer is yes. In general, say you have any norm $\| \cdot \|$ on $\mathbb{R}^p$. There is a measure $\mu$ on the boundary of the unit ball $B$ of $\| \cdot \|$, called the cone measure, with the property that there is the following version of integration in spherical coordinates: $$\int_{\mathbb{R}^p} f(x) \ dx = \int_0^\infty u^{p-1} \int_{\partial B} f(uy) \ d\mu(y) \ du$$ for any integrable function $f$.

Now in your case your density can be written in the form $f(x) = F(\| x \|)$, where $\| x \| = \sqrt{\sum (x_i/v_i)^2}$. This means that a random vector $X \sim N(0,V)$ has the property that $\| X \|$ and $X/\|X\|$ are independent, and the latter is distributed according to the cone measure on the surface of the ellipsoid $\{ x : \| x \|_V \le 1\}$.

• Yeah, the $v_i$'s should have been inverted. Fixed now. As you correctly inferred, I was using $||x|| = \sqrt(x'x)$, I made them into $||x||_2$ which is hopefully clearer. Thanks
– JMS
Commented Apr 14, 2011 at 20:44

Every nonnegative-definite symmetric real matrix is the matrix of covariances of the components of some random vector---that follows from the finite-dimensional spectral theorem. In Feller's terminology, the variance of a random vector $X$ is $E((X-\mu)(X-\mu)^T)$, where $\mu=E(X)$, so it is just the matrix of covariances. Now suppose $X$ is a random vector that is normally distributed with expected value $0$ and variance $M$, where $M$ is some positive-definite symmetric matrix. For now I'll assume $M$ is nonsingular. It is well-known that $M$ must have a positive-definite symmetric square root $M^{1/2}$. Then $M^{-1/2}X$ is normally distributed and its variance is the identity matrix. So as "independent quantities analogous to the direction and magnitude" of $X$ you could use the direction and magnitude of $M^{-1/2}X$. "Analogous to" is maybe a bit vague, so I don't know if that's the sort of thing you had in mind.

• Vague by intention :) I was sort of fishing. Initially I had taken $V$ diagonal (wlog up to a rotation of $x$). Here we arrive at the same place as @Mark Meckes generalized to any $V$. That is, taking $||x||_V^2 = x'V^{-1}x$ then $||x||_V$ and $x/||x||_V$ are independent when $x\sim N(0, V)$. Should probably dust off my linear models text!
– JMS
Commented Apr 16, 2011 at 1:16
• Dust off your LaTeX and MathJax skills too. Notice the typographical difference between $||a|| ||b||,$ coded as ||a|| ||b||, and $\|a\|\|b\|,$ coded as \|a\|\|b\|. The latter is considered correct. Also $\sqrt{x'x}$ is coded by using {curly braces}. Commented May 24 at 17:32