I am working with GP-UCB and need to calculate RKHS norm as in Theorem 6 of Srinivas et.al 2012. I found on page 3 column 1 like:

The induced RKHS norm $||{f}||_k=\sqrt{<f,f>}_k$ measures smoothness of $f$ w.r.t. $k$.

I am new to this field so cannot understand how to calculate $||f||_k$ numerically. Also, what does $<f,f>_k$ mean? How it is different from the conventional inner product $<f,f>$ ?

  • $\begingroup$ -1. This question should be asked on math.stackexchange.com instead of here which is a website for research-level mathematics. At least it should be posed there first before here. $\endgroup$ – Hans Oct 2 '19 at 7:01
  • 1
    $\begingroup$ The article references, as source for the concept and the notation, G. Wahba, Spline Models for Observational Data, Philadelphia, PA:SIAM, 1990. $\endgroup$ – Ben McKay Oct 3 '19 at 10:07

For a concise introduction to RKHS, you could have a look at sections 2.3 and 2.4 of Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences by Kanagawa et al. (2018).

In particular, they give a characterisation of the RKHS associated to a shift-invariant kernel on $\mathbb{R}^d$. In this case, the inner product is linked to the Fourier transform of the kernel:

Theorem 2.4 (Kanagawa et al.)

Let $k(x, y) = \Phi(x - y)$ be a shift-invariant kernel on $\mathbb{R}^d$ and assume $k \in C(\mathbb{R}^d) \cap L_1(\mathbb{R}^d)$.

Then the RKHS of k is given by

$$ \mathcal{H}_k = \Big\lbrace f \in C(\mathbb{R}^d) \cap L_1(\mathbb{R}^d), ~||f||_{\mathcal{H}_k} < \infty \Big\rbrace $$

Where the inner product is given by

$$ \langle f, g \rangle_{\mathcal{H}_k} = \frac{1}{(2\pi)^{d/2}}\int\frac{\mathcal{F}[f](\omega)\overline{\mathcal{F}[g](\omega)}}{\mathcal{F}[\Phi](\omega)}d\omega $$

If you're working with more exotic spaces or kernels (complex or bounded spaces, periodic kernels, ...) you could have a look at the book by Berlinet and Thomas-Agnan Reproducing Kernel Hilbert Spaces in Probability and Statistics.

Chapter 7.4 gives a list of RKHSs together with associated kernel and analytical expression for the inner product.

Finally, Chapter 6.2 mentions a numerical scheme for computing the RKHS norm in a special case. Maybe this could be interessting to you.

  • $\begingroup$ I am working with finite bounded space in $\mathbb{R}^n$ and with simple squared exponential kernels function. $\endgroup$ – Parikshit Pareek Oct 2 '19 at 7:24
  • $\begingroup$ Then example 2.7 in the Kanagawa paper is what you're looking for. $\endgroup$ – Cédric Travelletti Oct 2 '19 at 9:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.