So I'm trying to get the marginal density of a multivariate normal over an affine space if $A$ is a matrix in $\mathbb{R}^p \times \mathbb{R}^n$ for $p < n$ and $B \in \mathbb{R}^n$, $\Sigma$ is a positive definite matrix.

We're looking at... $$\lim_{\epsilon \rightarrow 0}\frac{1}{\epsilon}\int_{||Ax-B||<\epsilon} e^{-\frac{1}{2} x^t \Sigma^{-1} x } dx$$

Edit The space over which I integrate is wrong, see edit at the end... I should be controlling the distance to the projection, not something that depends on the scale of $A$ and $B$.

The first step is to get $$x_0 = \hbox{argmin}_x \left\[ x^t \Sigma^{-1} x | Ax-B = 0 \right\]$$

If one writes the Cholesky decomposition $\Sigma^-1 = L^T L$ and $\hbox{}^+$ is the Moore-Penrose pseudo inverse, then $$x0 = (A L^{-1})^{+} B$$

We substitute $x = x_0 + y$ in the integral, $y$ is orthogonal to $x_0$ with respect to $\Sigma^-1$ so the $x \Sigma^{-1} y$ terms disappear and we get something like

$$e^{-\frac{1}{2} x_0^t \Sigma^{-1} x_0} \lim_{\epsilon \rightarrow 0}\frac{1}{\epsilon}\int_{||Ay||<\epsilon} e^{-\frac{1}{2} y^t \Sigma^{-1} y} dy$$

setting $z = Ly$

Edit urrrr... $$e^{-\frac{1}{2} x_0^t \Sigma^{-1} x_0} \lim_{\epsilon \rightarrow 0}\frac{1}{\epsilon}\int_{||A L^{-1}z||<\epsilon} e^{-\frac{1}{2} z^t z} \frac{dy}{dz} dz$$

hum the $\frac{dy}{dz}$ wouldn't be very nice...

At this point, I've gotten rid of the affine part by factoring the $x_0$ out ( yay.. ) but I can't quite get that last cut... It's likely going to involve the determinant of $\Sigma$ and the rank of $A$ ( assume it's $p$ if needed ) but I can't figure out how.

a) Do you notice anything obviously wrong in the derivation of $x_0$ ?

b) What's the obviously right answer that has been eluding me ?

Edit c) Possible way, orthonormalize the kernel of $A$ then the rest of the space. Then the problem boils down to finding the density of a multivariate normal conditional on some elements of the vector being known... that's not a nice formula, it involves slicing $\Sigma$ and taking the Schur complement see Wikipedia, Multivariate normal, Conditional distributions

Edit It occurs to me that the $||Ay||<\epsilon$ criterion isn't the right one, otherwise, the answer would depend on the scale of $A$ which is dumb. I guess the right criterion is $||A^{t}(A A^{t})^{-1}A y||<\epsilon$, which is the distance of vector $y$ to its orthogonal projection on the subspace $A. = $



1 Answer 1


I had to do something similar recently and there's a neat trick for integrating Gaussian over linear subspace of $\mathbb{R}^n$ -- write it as an integral over whole $\mathbb{R}^n$ but substitute Dirac measure for regular. Here's overview, and discussion on math.overflow


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.