1
$\begingroup$

Consider the following exponential kernel:

$k(x_1, x_2) = \exp\left(\frac{|x_1 - x_2|}{L}\right)$,

which is symmetric and non-negative definite. By virtue of Mercer's theorem, we have

$k(x_1, x_2) = \sum_{i = 1}^\infty \lambda_i \phi_i(x_1) \phi_i(x_2)$

where $\lambda_i$ and $\phi_i$ are the eigenvalues and eigenfunctions of $k$, respectively. Now, consider the following product:

$K((x_1, y_1), (x_2, y_2)) := k(x_1, x_2) k(y_1, y_2) = \exp\left( -\frac{|x_1 - x_2|}{L} - \frac{|y_1 - y_2|}{L}\right)$.

Since the product of two symmetric, non-negative definite kernels is another kernel with the same properties, Mercer's theorem still applies.

The question is: Having computed $\lambda_i$ and $\phi_i$ of $k$, what can we say about the eigenfunctions and eigenvalues of $K$?

Thank you.

Regards, Ivan

$\endgroup$
2
  • $\begingroup$ Thank you for the quick reply. Please have a look at venus.usc.edu/book/chp2/node13.html#SECTION00133200000000000000, starting from equation 1.39, and at venus.usc.edu/book/chp5/node11.html#SECTION00131100000000000000. The author assumes the product structure for the eigenvalues and eigenfunctions of $K$ with respect to the eigenvalues and eigenfunctions of $k$ (following the notation in my question). I just do not see what this assumption is based on, and I though I would get some kind of a confirmation here. Can you please comment on those derivations? Thank you. $\endgroup$
    – Ivan
    Jan 13, 2013 at 13:52
  • $\begingroup$ I removed the comment because it was misleading... $\endgroup$
    – Suvrit
    Jan 13, 2013 at 17:44

1 Answer 1

2
$\begingroup$

You have two independent sets of variables, so it is a tensor product, not a Hadamard product, no? I am not sure I understand Suvrit's comment...

I think you just get

$K((x_1,y_1),(x_2,y_2)) = k(x_1,x_2)k(y_1,y_2) = \sum_{i=1}^\infty \lambda_i \phi_i(x_1)\phi_i(x_2)\sum_{j=1}^\infty \lambda_j \phi_j(y_1)\phi_j(y_2)$

$=\sum_{i,j=1}^\infty \lambda_i \lambda_j \phi_i(x_1)\phi_j(y_1) \phi_i(x_2)\phi_j(y_2)$

so that the eigenvalues of $K$ are just products of the eigenvalues of $k$,

$\Lambda_{i,j} = \lambda_i \lambda_j$.

And the eigenfunctions are simply products, too, i.e.

$\Phi_{i,j}(x,y) = \phi_i(x)\phi_j(y)$.

If you want sums over one index, you can now use your favorite enumeration of $N\times $,

$\Lambda_1 = \lambda_1 \lambda_1$,

$\Lambda_2 = \lambda_1 \lambda_2$, $\Lambda_3 = \lambda_2 \lambda_1$,

$\Lambda_4 = \lambda_1 \lambda_3$, $\Lambda_5 = \lambda_2 \lambda_2$, $\Lambda_6 = \lambda_3 \lambda_1$,

etc. (and similarly for the $\Phi$'s).

$\endgroup$
6
  • $\begingroup$ Your explanation seems to be clarifying everything. Thank you. The only thing that I am worrying about is the repeating eigenvalues $\lambda_i \lambda_j$ and $\lambda_j \lambda_i$, $i \neq j$. Should not they and the corresponding eigenfunctions be merged? Referring to the same source as before (venus.usc.edu/book/chp5/node11.html#SECTION00131100000000000000), it is suggested in equation 1.60 that the expression should be $\Phi_{ij}(x, y) = \frac{1}{\sqrt{2}}(\phi_i(x)\phi_j(y) + \phi_j(x)\phi_i(y))$, but I am not sure how to get there. What do you think? Thank you. $\endgroup$
    – Ivan
    Jan 13, 2013 at 17:20
  • $\begingroup$ oh, I guess I interpreted it to be a Hadamard product, not the tensor Kronecker product --- should have looked more carefully. So I'll delete my misleading comment. $\endgroup$
    – Suvrit
    Jan 13, 2013 at 17:44
  • 1
    $\begingroup$ Products $\lambda_i\lambda_j$, $i\not=j$, occur always twice, i.e., they have have two eigenfunctions, $\phi_i(x)\phi_j(y)$ and $\phi_i(y)\phi_j(x)$. If you want, you can pass to another basis of this eigenspace, e.g., taking the symmetric and the anti-symmetric functions: $\frac{1}{\sqrt{2}} \big(phi_i(x)\phi_j(y) + \phi_i(y)\phi_j(x)\big)$ $\frac{1}{\sqrt{2}} \big(phi_i(x)\phi_j(y) - \phi_i(y)\phi_j(x)\big)$ $\endgroup$
    – Uwe Franz
    Jan 13, 2013 at 18:17
  • $\begingroup$ Does it mean that the construction from the reference above is invalid? Since you have this second part with the difference, which is not present there. How did you come up with such a construction? Can you please give me some directions where I can read about it? Thank you. $\endgroup$
    – Ivan
    Jan 13, 2013 at 18:27
  • $\begingroup$ I am not sure, maybe they have some reason why they are only interested in the symmetric solution? $\endgroup$
    – Uwe Franz
    Jan 13, 2013 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.