Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Consider the following exponential kernel:

$k(x_1, x_2) = \exp\left(\frac{|x_1 - x_2|}{L}\right)$,

which is symmetric and non-negative definite. By virtue of Mercer's theorem, we have

$k(x_1, x_2) = \sum_{i = 1}^\infty \lambda_i \phi_i(x_1) \phi_i(x_2)$

where $\lambda_i$ and $\phi_i$ are the eigenvalues and eigenfunctions of $k$, respectively. Now, consider the following product:

$K((x_1, y_1), (x_2, y_2)) := k(x_1, x_2) k(y_1, y_2) = \exp\left( -\frac{|x_1 - x_2|}{L} - \frac{|y_1 - y_2|}{L}\right)$.

Since the product of two symmetric, non-negative definite kernels is another kernel with the same properties, Mercer's theorem still applies.

The question is: Having computed $\lambda_i$ and $\phi_i$ of $k$, what can we say about the eigenfunctions and eigenvalues of $K$?

Thank you.

Regards, Ivan

share|improve this question
    
Thank you for the quick reply. Please have a look at venus.usc.edu/book/chp2/node13.html#SECTION00133200000000000000, starting from equation 1.39, and at venus.usc.edu/book/chp5/node11.html#SECTION00131100000000000000. The author assumes the product structure for the eigenvalues and eigenfunctions of $K$ with respect to the eigenvalues and eigenfunctions of $k$ (following the notation in my question). I just do not see what this assumption is based on, and I though I would get some kind of a confirmation here. Can you please comment on those derivations? Thank you. –  Ivan Jan 13 '13 at 13:52
    
I removed the comment because it was misleading... –  Suvrit Jan 13 '13 at 17:44
add comment

1 Answer

up vote 2 down vote accepted

You have two independent sets of variables, so it is a tensor product, not a Hadamard product, no? I am not sure I understand Suvrit's comment...

I think you just get

$K((x_1,y_1),(x_2,y_2)) = k(x_1,x_2)k(y_1,y_2) = \sum_{i=1}^\infty \lambda_i \phi_i(x_1)\phi_i(x_2)\sum_{j=1}^\infty \lambda_j \phi_j(y_1)\phi_j(y_2)$

$=\sum_{i,j=1}^\infty \lambda_i \lambda_j \phi_i(x_1)\phi_j(y_1) \phi_i(x_2)\phi_j(y_2)$

so that the eigenvalues of $K$ are just products of the eigenvalues of $k$,

$\Lambda_{i,j} = \lambda_i \lambda_j$.

And the eigenfunctions are simply products, too, i.e.

$\Phi_{i,j}(x,y) = \phi_i(x)\phi_j(y)$.

If you want sums over one index, you can now use your favorite enumeration of $N\times $,

$\Lambda_1 = \lambda_1 \lambda_1$,

$\Lambda_2 = \lambda_1 \lambda_2$, $\Lambda_3 = \lambda_2 \lambda_1$,

$\Lambda_4 = \lambda_1 \lambda_3$, $\Lambda_5 = \lambda_2 \lambda_2$, $\Lambda_6 = \lambda_3 \lambda_1$,

etc. (and similarly for the $\Phi$'s).

share|improve this answer
    
Your explanation seems to be clarifying everything. Thank you. The only thing that I am worrying about is the repeating eigenvalues $\lambda_i \lambda_j$ and $\lambda_j \lambda_i$, $i \neq j$. Should not they and the corresponding eigenfunctions be merged? Referring to the same source as before (venus.usc.edu/book/chp5/node11.html#SECTION00131100000000000000), it is suggested in equation 1.60 that the expression should be $\Phi_{ij}(x, y) = \frac{1}{\sqrt{2}}(\phi_i(x)\phi_j(y) + \phi_j(x)\phi_i(y))$, but I am not sure how to get there. What do you think? Thank you. –  Ivan Jan 13 '13 at 17:20
    
oh, I guess I interpreted it to be a Hadamard product, not the tensor Kronecker product --- should have looked more carefully. So I'll delete my misleading comment. –  Suvrit Jan 13 '13 at 17:44
1  
Products $\lambda_i\lambda_j$, $i\not=j$, occur always twice, i.e., they have have two eigenfunctions, $\phi_i(x)\phi_j(y)$ and $\phi_i(y)\phi_j(x)$. If you want, you can pass to another basis of this eigenspace, e.g., taking the symmetric and the anti-symmetric functions: $\frac{1}{\sqrt{2}} \big(phi_i(x)\phi_j(y) + \phi_i(y)\phi_j(x)\big)$ $\frac{1}{\sqrt{2}} \big(phi_i(x)\phi_j(y) - \phi_i(y)\phi_j(x)\big)$ –  Uwe Franz Jan 13 '13 at 18:17
    
Does it mean that the construction from the reference above is invalid? Since you have this second part with the difference, which is not present there. How did you come up with such a construction? Can you please give me some directions where I can read about it? Thank you. –  Ivan Jan 13 '13 at 18:27
    
I am not sure, maybe they have some reason why they are only interested in the symmetric solution? –  Uwe Franz Jan 13 '13 at 20:31
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.