Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Hello guys, I am taking a look at the "Gaussian Process Latent Variable Models" (GP-LVM) from Lawrence's paper (LINK) and because of my lack of experience in Bayesian modeling I am having some trouble with his derivation; I hope you can help :)

Basic formulation

Following the paper, $\mathbf{Y}=[\mathbf{y}_1, ..., \mathbf{y}_n]^T$ is the observations matrix and similarly $\mathbf{X}$ is the matrix of latent variables. The distributions are gaussian and defined as

$$p(\mathbf{y}_n | \mathbf{x}_n, \mathbf{W}, \beta) = N(\mathbf{y}_n | \mathbf{W}\mathbf{x}_n, \beta^{-1}\mathbf{I}) \quad \quad p(\mathbf{x}_n)=N(\mathbf{x}_n|0,\mathbf{I})$$

Assuming i.i.d. his the given relationship in between latent space and observations is:

$$p(\mathbf{Y} | \mathbf{W},\beta) = \prod_{n=1}^N p(\mathbf{Y}_n | \mathbf{W})$$

What I am interested in understanding is how he obtains Eq.(1), that is, the expression for $p(\mathbf{Y}|\mathbf{X},\beta)$:

$$p(\mathbf{Y}|\mathbf{X},\beta) = \text{(scale)} \: \text{exp} \left( -\frac{1}{2} tr( \mathbf{K}^{-1} \mathbf{Y} \mathbf{Y}^T) \right), \quad \mathbf{K}=\alpha \mathbf{XX}^T + \beta^{-1}\mathbf{I} $$

Note that to achieve this he assumes that the linear map $\mathbf{W}$ is distributed as $$p(W)=\prod_{i=1}^{D} N(\mathbf{w}_i | 0, \alpha^{-1}\mathbf{I})$$

Begin of self-explanation

What I believe he's doing is first removing the conditional relationship w.r.t. $\mathbf{W}$ by:

$$p(\mathbf{Y},\mathbf{W}|\beta) = p(\mathbf{Y}|\mathbf{W},\beta) p(\mathbf{W})$$

Then, he marginalizes w.r.t. $\mathbf{W}$ to get rid of the dependence from the linear mapping:

$$p(\mathbf{Y} | \mathbf{X}, \beta) = \int_{\mathbf{W}} p(\mathbf{Y},\mathbf{W}|\beta) $$

Now, if what I have above is correct, my simplified question becomes:

$$ \mathbf{\text{how do you integrate w.r.t. a matrix?}} $$

share|improve this question
add comment

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.