Skip to main content
2 of 3
clarify definition
Emile
  • 183
  • 5

RKHS norm and posterior of Gaussian process

In Srinivas et al (2010) [appendix B], the authors claim the following "easy to see" property relating the norm of a function in a RKHS of kernel $k(\cdot,\cdot)$, and its norm in the RKHS built with the posterior covariance of a Gaussian process with the same kernel $k(\cdot,\cdot)$. I'm unable to see why this claim is correct:

Definitions: Let $k:\mathcal{X} \times \mathcal{X} \to \mathbb{R}^+$ be a kernel function and $\mathcal{H}_k$ the associated RKHS. Let $x_1, \dots, x_T$ be a finite sequence of points in $\mathcal{X}$. The posterior covariance function of a Gaussian process of kernel $k$ perturbed by independent $\mathcal{N}(0,\sigma^2)$ noise is: $$k_T(x,y) := k(x,y) - \mathrm{k}_T^\top(x)\Big(\mathrm{K}_T+\sigma^2\mathrm{I}\Big)^{-1}\mathrm{k}_T(y)\,,$$ where the $(T\times 1)$-vector $\mathrm{k}_T$ is defined by $\big[\mathrm{k}_T(x)\big]_t := k(x_t,x)$ and the $(T\times T)$-matrix $\mathrm{K}_T$ by $\big[\mathrm{K}_T\big]_{t,t'} := k(x_t,x_{t'})$.
Let $\mathcal{H}_{k_T}$ be the associated RKHS.

Question: Show that $\mathcal{H}_k = \mathcal{H}_{k_T}$ and for all $f\in \mathcal{H}_k$: $$\|f\|_{k_T}^2 = \|f\|_k^2 + \sigma^{-2} \sum_{t=1}^T f(x_t)^2\,.$$

Emile
  • 183
  • 5