Let $c(i,i')$ be a kernel function on a reasonable index space $I$. Choose a dense sequence of points $\{i_1, i_2, \cdots \} \subseteq I$, and define the one-point kernel functions $k_n := c(\cdot, i_n)$. Let $U$ be the Hilbert span of these functions; by the reproducing kernel property, $\langle k_n, k_{n'} \rangle := c(i_n, i_{n'})$.

Using the Gram-Schmidt procedure, we convert the sequence $\{k_n\}$ into an orthogonal basis $\{\hat k_n\}$, by subtracting away the projection onto the previous axes (without normalization). Define the "novel variance" $\hat \sigma_n^2 := \langle \hat k_n, \hat k_n \rangle$, which represents the new contribution to variance from data located at $i_n$.

Now, define $\Sigma^2 := \sum \hat \sigma_n^2$, which represents all the possible novel variance to be found in $I$. It is not difficult to show that $\Sigma^2$ does not depend on the choice of basis.

Question: If $I$ is compact, then is $\Sigma^2 < \infty$?


closed as unclear what you're asking by Tom LaGatta, Chris Godsil, Stefan Kohl, user9072, Noah Stein Apr 16 '14 at 18:50

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ What do you mean by "doesn't depend on the choice of basis"? It seems that you are not choosing a basis. $\endgroup$ – Alexander Shamov Apr 3 '14 at 0:20
  • $\begingroup$ On the other hand, it does depend on the order in which you take the $i_n$'s. For instance, even for 2 vectors, say, $i_1$ and $i_2$, your sum of squared norms will be $\Vert i_1 \Vert^2 + \Vert i_2 \Vert^2 - \frac{\langle i_1,i_2 \rangle}{\Vert i_1 \Vert^2}$, which is not symmetric in $i_1,i_2$. $\endgroup$ – Alexander Shamov Apr 3 '14 at 0:22
  • $\begingroup$ Thanks, Alexander. I clearly have screwed something up in the definition. The 2 vector case is supposed to be $\|i_1\|^2 + \|i_2\|^2 - \langle i_1, i_2 \rangle$, which is symmetric. $\endgroup$ – Tom LaGatta Apr 3 '14 at 0:40
  • $\begingroup$ Then I don't have a guess what you are computing in the case of more than 2 vectors. $\endgroup$ – Alexander Shamov Apr 3 '14 at 0:46
  • $\begingroup$ Voting to close my own question. $\endgroup$ – Tom LaGatta Apr 3 '14 at 1:02

You haven't specified what role is played by the topology of $I$, so my default assumption will be that the kernel is continuous, in which case the answer to your question is negative.

For a counterexample, let $I$ be the one-point compactification of $\mathbb{N}$, that is, $\{1,2,\dots,\infty\}$, and take $k_{nm} := \delta_{nm} n^{-1}$. Then the basis is already orthogonal, so $\hat\sigma_n^2 = k_{nn}$, and $\sum_n \hat\sigma_n^2 = \infty$.

  • $\begingroup$ Thanks for the counterexample. As your comments show the question was ill-formed. I'm marking this as an answer to give you the points in gratitude for your observation. $\endgroup$ – Tom LaGatta Apr 3 '14 at 1:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.