2
$\begingroup$

Suppose that $x\sim N(0, V)$ is $p$ dimensional with $V$ diagonal having elements $v_i^2$. Then

\begin{align} f_x(x) & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{x^2_i}{v_i^2}\right) \\[10pt] & \propto \left(\prod_p v_i\right)^{-1} \exp\left(-\frac{1}{2}\sum_p \frac{\|x\|_2^2x^2_i}{\|x\|_2^2v_i^2}\right) \end{align}

Now let $y_i = x_i/\|x\|_2$ and $u=\|x\|_2$. Making the transformation gives

$$f_{u,y}(u,y) \propto \left(\prod_p v_i\right)^{-1} u^{p-1}\exp\left(-\frac{u^2}{2} \sum_p \frac{y^2_i}{v_i^2}\right)$$

where $u\in (0, \infty)$ and $y'y=1$ (with $u^{p-1}$ coming in through the Jacobian). The density doesn't factor (unless $V\propto I$), so $u$ and $y$ are dependent. This is perfectly sensible to me; informally, in the 2-dimensional case if $V=\operatorname{diag}(10000, 1)$ then clearly if the direction is near $(1,0)$ the magnitude will be larger than if it were near $(0,1)$. Similarly, it's intuitive that the dependence disappears if $V \propto I$ (in which case $y$ falls out of the density entirely).

My question is as follows: First, is my reasoning (and math!) correct? Second, in the first case where $V\not \propto I$ is it possible to reparameterize in terms of independent quantities analogous to the direction and magnitude ( maybe something like, for example, requiring $y$ to lie on an ellipsoid determined by $V$)? It seems like there should be but it's eluding me.

$\endgroup$
1
  • $\begingroup$ The reasoning looks correct. I think you can re-parametrize by keeping the direction $y$ the same, and just dividing $x$ by the standard deviation (point-wise, that is, $x_i' = x_i/v_i$) in $u$, that is, defining $u$ as $u = || (x_1 / v_1, .., x_p/v_p) ||$. B.t.w. isn't $f_x(x)$ proportional to $1/\prod_i v_i$ (instead of $\prod_p v_i$)? $\endgroup$
    – Or Zuk
    Apr 14, 2011 at 19:06

2 Answers 2

5
$\begingroup$

Your reasoning looks right, although I'm not that familiar with the exact notation you're using, except that the $v_i$ should be in the denominator, not the numerator.

In the second case the answer is yes. In general, say you have any norm $\| \cdot \|$ on $\mathbb{R}^p$. There is a measure $\mu$ on the boundary of the unit ball $B$ of $\| \cdot \|$, called the cone measure, with the property that there is the following version of integration in spherical coordinates: $$ \int_{\mathbb{R}^p} f(x) \ dx = \int_0^\infty u^{p-1} \int_{\partial B} f(uy) \ d\mu(y) \ du $$ for any integrable function $f$.

Now in your case your density can be written in the form $f(x) = F(\| x \|)$, where $\| x \| = \sqrt{\sum (x_i/v_i)^2}$. This means that a random vector $X \sim N(0,V)$ has the property that $\| X \|$ and $X/\|X\|$ are independent, and the latter is distributed according to the cone measure on the surface of the ellipsoid $\{ x : \| x \|_V \le 1\}$.

$\endgroup$
1
  • 1
    $\begingroup$ Yeah, the $v_i$'s should have been inverted. Fixed now. As you correctly inferred, I was using $||x|| = \sqrt(x'x)$, I made them into $||x||_2$ which is hopefully clearer. Thanks $\endgroup$
    – JMS
    Apr 14, 2011 at 20:44
2
$\begingroup$

Every nonnegative-definite symmetric real matrix is the matrix of covariances of the components of some random vector---that follows from the finite-dimensional spectral theorem. In Feller's terminology, the variance of a random vector $X$ is $E((X-\mu)(X-\mu)^T)$, where $\mu=E(X)$, so it is just the matrix of covariances. Now suppose $X$ is a random vector that is normally distributed with expected value $0$ and variance $M$, where $M$ is some positive-definite symmetric matrix. For now I'll assume $M$ is nonsingular. It is well-known that $M$ must have a positive-definite symmetric square root $M^{1/2}$. Then $M^{-1/2}X$ is normally distributed and its variance is the identity matrix. So as "independent quantities analogous to the direction and magnitude" of $X$ you could use the direction and magnitude of $M^{-1/2}X$. "Analogous to" is maybe a bit vague, so I don't know if that's the sort of thing you had in mind.

$\endgroup$
1
  • $\begingroup$ Vague by intention :) I was sort of fishing. Initially I had taken $V$ diagonal (wlog up to a rotation of $x$). Here we arrive at the same place as @Mark Meckes generalized to any $V$. That is, taking $||x||_V^2 = x'V^{-1}x$ then $||x||_V$ and $x/||x||_V$ are independent when $x\sim N(0, V)$. Should probably dust off my linear models text! $\endgroup$
    – JMS
    Apr 16, 2011 at 1:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.