I've been stuck with that statement as well in the past, especially regarding the RKHS norm equation. There are different ways to prove it, but one that involves just simple linear algebra and feature maps is the following.

Let $\phi: \mathcal{X} \to \mathcal{H}_k$ denote the canonical feature map, i.e., $\phi(x) = k(\cdot, x)$, for $x \in \mathcal{X}$. Now define an operator $\mathrm{W}_t:\mathcal{H}_k\to\mathcal{H}_k$ as:
\begin{equation}
\mathrm{W}_t := \sigma^{-2}\Phi_t\Phi_t^\mathtt{T} + I,
\end{equation}
where $\Phi_t := [\phi(x_1), \dots, \phi(x_t)]: \mathbb{R}^t\to\mathcal{H}_k$ and $A^\mathtt{T}$ denotes the transpose or adjoint of operator $A$. Note that in general we can treat $\mathrm{W}_t$ as a (infinite-dimensional) matrix. Since $\mathrm{W}_t$ is positive definite, it has an inverse. Applying Woodbury's identity yields:
\begin{equation}
\mathrm{W}_t^{-1} = (\sigma^{-2}\Phi_t\Phi_t^\mathtt{T} + I)^{-1} = I - \Phi_t(\sigma^2 I + \Phi_t^\mathtt{T}\Phi_t)^{-1}\Phi_t^\mathtt{T} = I - \Phi_t(\mathrm{K}_t + \sigma^2 I)^{-1}\Phi_t^\mathtt{T} \,.
\end{equation}
We then see that the GP posterior kernel $k_t$ can be obtained as:
\begin{equation}
\begin{split}
\langle \phi(x), \mathrm{W}_t^{-1} \phi(x') \rangle_k &= \langle \phi(x), \phi(x') \rangle_k - \phi(x)^\mathtt{T} \Phi_t(\mathrm{K}_t + \sigma^2 I)^{-1}\Phi_t^\mathtt{T}\phi(x')\\
&= k(x,x') - \mathrm{k}_t(x)^\mathtt{T} \Phi_t(\mathrm{K}_t + \sigma^2 I)^{-1}\mathrm{k}_t(x')\\
&= k_t(x,x')\,.
\end{split}
\end{equation}

Now observe that $\mathrm{W}_t^{-1/2} \phi : \mathcal{X} \to \mathcal{H}_k$ defines a (non-canonical) feature map for $k_t$, using $\mathcal{H}_k$ as the feature space, since $\langle \mathrm{W}_t^{-1/2} \phi(x), \mathrm{W}_t^{-1/2} \phi(x') \rangle_k = \langle \phi(x), \mathrm{W}_t^{-1} \phi(x') \rangle_k = k_t(x,x')$. By the reproducing property, for any $f \in \mathcal{H}_{k}$, we have that:
\begin{equation}
\begin{split}
\forall x \in \mathcal{X}, \quad f(x) = \langle f, \phi(x) \rangle_k &= \langle f, \mathrm{W}_t^{1/2} \mathrm{W}_t^{-1/2} \phi(x) \rangle_k\\
&= \langle \mathrm{W}_t^{1/2} f, \mathrm{W}_t^{-1/2} \phi(x) \rangle_k\\
&= \langle f, k_t(\cdot, x) \rangle_{k_t}
\end{split}
\end{equation}
Therefore, $\mathrm{W}_t^{1/2} f$ is the representation of $f \in \mathcal{H}_{k_t}$ in the feature space. The norm of $f$ is then given by:
\begin{equation}
\begin{split}
\lVert f \rVert_{k_t}^2 &= \langle \mathrm{W}_t^{1/2} f, \mathrm{W}_t^{1/2} f \rangle_k\\
&= \langle f, \mathrm{W}_t f \rangle_k\\
&= f^\mathtt{T} (\sigma^{-2}\Phi_t\Phi_t^\mathtt{T} + I) f\\
&= \lVert f \rVert_k^2 + \sigma^{-2} f^\mathtt{T} \Phi_t\Phi_t^\mathtt{T} f\\
&= \lVert f \rVert_k^2 + \sigma^{-2} \sum_{i=1}^t f(x_i)^2\,,
\end{split}
\end{equation}
since $\Phi_t^\mathtt{T} f = [f(x_1), \dots, f(x_t)]^\mathtt{T} \in \mathbb{R}^t$.

To prove that $\mathcal{H}_k = \mathcal{H}_{k_t}$, I'm not sure what is the easiest way, but I think one could show that (1) any $f \in \mathcal{H}_k$ has a bounded norm in $\mathcal{H}_{k_t}$, so that $\mathcal{H}_k \subset \mathcal{H}_{k_t}$, and (2) the difference $k - k_t$ defines a positive-semidefinite kernel, which satisfies the main condition for $\mathcal{H}_{k_t} \subset \mathcal{H}_k$ (see, for example, Berlinet & Thomas-Agnan, 2012, sec. 4.5, Thr. 12).

References:

- Berlinet, A., & Thomas-Agnan, C. (2004). Reproducing Kernel Hilbert Spaces in Probability and Statistics. In Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer. https://doi.org/10.1007/978-1-4419-9096-9