Question

So I'm trying to get the marginal density of a multivariate normal over an affine space if $A$ is a matrix in $\mathbb{R}^p \times \mathbb{R}^n$ for $p < n$ and $B \in \mathbb{R}^n$, $\Sigma$ is a positive definite matrix.

We're looking at... $$\lim_{\epsilon \rightarrow 0}\frac{1}{\epsilon}\int_{||Ax-B||<\epsilon} e^{-\frac{1}{2} x^t \Sigma^{-1} x } dx$$

Edit The space over which I integrate is wrong, see edit at the end... I should be controlling the distance to the projection, not something that depends on the scale of $A$ and $B$.

The first step is to get $$x_0 = \hbox{argmin}_x \left[ x^t \Sigma^{-1} x | Ax-B = 0 \right]$$

If one writes the Cholesky decomposition $\Sigma^-1 = L^T L$ and $\hbox{}^+$ is the Moore-Penrose pseudo inverse, then $$x0 = (A L^{-1})^{+} B$$

We substitute $x = x_0 + y$ in the integral, $y$ is orthogonal to $x_0$ with respect to $\Sigma^-1$ so the $x \Sigma^{-1} y$ terms disappear and we get something like

$$e^{-\frac{1}{2} x_0^t \Sigma^{-1} x_0} \lim_{\epsilon \rightarrow 0}\frac{1}{\epsilon}\int_{||Ay||<\epsilon} e^{-\frac{1}{2} y^t \Sigma^{-1} y} dy$$

setting $z = Ly$

Edit urrrr... $$e^{-\frac{1}{2} x_0^t \Sigma^{-1} x_0} \lim_{\epsilon \rightarrow 0}\frac{1}{\epsilon}\int_{||A L^{-1}z||<\epsilon} e^{-\frac{1}{2} z^t z} \frac{dy}{dz} dz$$

hum the $\frac{dy}{dz}$ wouldn't be very nice...

At this point, I've gotten rid of the affine part by factoring the $x_0$ out ( yay.. ) but I can't quite get that last cut... It's likely going to involve the determinant of $\Sigma$ and the rank of $A$ ( assume it's $p$ if needed ) but I can't figure out how.

a) Do you notice anything obviously wrong in the derivation of $x_0$ ?

b) What's the obviously right answer that has been eluding me ?

Edit c) Possible way, orthonormalize the kernel of $A$ then the rest of the space. Then the problem boils down to finding the density of a multivariate normal conditional on some elements of the vector being known... that's not a nice formula, it involves slicing $\Sigma$ and taking the Schur complement see Wikipedia, Multivariate normal, Conditional distributions

Edit It occurs to me that the $||Ay||<\epsilon$ criterion isn't the right one, otherwise, the answer would depend on the scale of $A$ which is dumb. I guess the right criterion is $||A^{t}(A A^{t})^{-1}A y||<\epsilon$, which is the distance of vector $y$ to its orthogonal projection on the subspace $A. = $

Thanks!

Answer 1

I had to do something similar recently and there's a neat trick for integrating Gaussian over linear subspace of $\mathbb{R}^n$ -- write it as an integral over whole $\mathbb{R}^n$ but substitute Dirac measure for regular. Here's overview, and discussion on math.overflow