Dependence between direction and magnitude of multivariate normal random vector

Question

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

\begin{align} f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2}\right) \\[10pt] & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{\|x\|_2^2x^2_i}{\|x\|_2^2v_i^2}\right) \end{align}

Now let $y_i = x_i/\|x\|_2$ and $u=\|x\|_2$. Making the transformation gives

$$f_{u,y}(u,y) \propto \left(\prod_p v_i\right)^{-1} u^{p-1}\exp\left(-\frac{u^2}{2} \sum_p \frac{y^2_i}{v_i^2}\right)$$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2-dimensional case if $V=\operatorname{diag}(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

Answer 1

Your reasoning looks right, although I'm not that familiar with the exact notation you're using, except that the $v_i$ should be in the denominator, not the numerator.

In the second case the answer is yes. In general, say you have any norm $\| \cdot \|$ on $\mathbb{R}^p$. There is a measure $\mu$ on the boundary of the unit ball $B$ of $\| \cdot \|$, called the cone measure, with the property that there is the following version of integration in spherical coordinates: $$ \int_{\mathbb{R}^p} f(x) \ dx = \int_0^\infty u^{p-1} \int_{\partial B} f(uy) \ d\mu(y) \ du $$ for any integrable function $f$.

Now in your case your density can be written in the form $f(x) = F(\| x \|)$, where $\| x \| = \sqrt{\sum (x_i/v_i)^2}$. This means that a random vector $X \sim N(0,V)$ has the property that $\| X \|$ and $X/\|X\|$ are independent, and the latter is distributed according to the cone measure on the surface of the ellipsoid $\{ x : \| x \|_V \le 1\}$.

Answer 2

Every nonnegative-definite symmetric real matrix is the matrix of covariances of the components of some random vector---that follows from the finite-dimensional spectral theorem. In Feller's terminology, the variance of a random vector $X$ is $E((X-\mu)(X-\mu)^T)$, where $\mu=E(X)$, so it is just the matrix of covariances. Now suppose $X$ is a random vector that is normally distributed with expected value $0$ and variance $M$, where $M$ is some positive-definite symmetric matrix. For now I'll assume $M$ is nonsingular. It is well-known that $M$ must have a positive-definite symmetric square root $M^{1/2}$. Then $M^{-1/2}X$ is normally distributed and its variance is the identity matrix. So as "independent quantities analogous to the direction and magnitude" of $X$ you could use the direction and magnitude of $M^{-1/2}X$. "Analogous to" is maybe a bit vague, so I don't know if that's the sort of thing you had in mind.