The error is in this line:

The standard argument shows that $\widetilde{\mathcal{M}}$ is an RKHS of functions on $X$.

In fact, this is not generally true. The completion $\widetilde{\mathcal{M}}$ may not be naturally identified with a space of functions on $X$.

The "obvious" way that one would try to prove this is as follows. Consider an element $\phi \in \widetilde{\mathcal{M}}$. Since $\mathcal{M}$ is dense in its completion, there is a sequence $\{f_n\} \subset \mathcal{M}$ such that $f_n \to \phi$ in $\widetilde{\mathcal{M}}$-norm, which is an extension of the $\mathcal{M}$-norm. In particular, $\{f_n\}$ is $\mathcal{M}$-norm Cauchy. Because of your inequality $|f(x)| \le \|f\|_{\mathcal{M}} \sqrt{K(x,x)}$ (*), we have that $\{f_n(x)\}$ is Cauchy in $\mathbb{R}$ for each $x$. So $\{f_n(x)\}$ converges to a number which we may call $f_\phi(x)$; that is, $f_n \to f_\phi$ pointwise. It is also easy to show that $f_{\phi}$ does not depend on the choice of sequence $f_n \to \phi$, so the linear map $\phi \mapsto f_\phi$ of $\widetilde{\mathcal{M}}$ into $\mathbb{R}^X$ is well defined. Let's call this map $T$.

So the "obvious" thing to do is to look at the image of $T$, which is of course a function space, and make it a Hilbert space by pushing forward the $\widetilde{\mathcal{M}}$-norm. If this works, then the same inequality (*) will show that the evaluation map is continuous in this norm, and we have ourselves an RKHS.

The problem is that $T$ might fail to be injective. In other words, we could have a nonzero $\phi$ for which $f_\phi = 0$. In that case, pushing forward the norm will not work; it will not be well defined on the image of $T$.

This is exactly what happens in the counterexample you discuss in the comments. As you suggest, consider $\ell^2$ as an RKHS on $X = \mathbb{N} = \{1,2,\dots\}$, with its usual orthonormal basis $\{e_n\}$, and let $v = \sum_n \frac{1}{n} e_n$. Let $H = \{v\}^\perp$ with the same $\ell^2$ inner product; being a closed subspace of $\ell^2$, $H$ is an RKHS.

Let $P : \ell^2 \to H$ be the orthogonal projection and let $g_n = P e_n$. Following your construction, let $\mathcal{M} \subset H$ be the linear span of $\{g_n\}$ and let $\langle\cdot, \cdot\rangle_{\mathcal{M}}$ be the inner product on $\mathcal{M}$ which makes the $\{g_n\}$ orthonormal.

Set $h_m = \sum_{n=1}^m \frac{1}{n} g_n$. Clearly the sequence $\{h_m\}$ is Cauchy in $\mathcal{M}$, so it converges in the completion $\widetilde{\mathcal{M}}$ to some $\phi$. It is also clear that $\|\phi\|_\widetilde{\mathcal{M}}^2 = \frac{\pi^2}{6}$ so in particular $\phi \ne 0$.

But on the other hand, we have $h_m = \sum_{n=1}^m \frac{1}{n} P e_n = P\left(\sum_{n=1}^m \frac{1}{n} e_n\right)$. By continuity of $P$, we have $h_m \to Pv = 0$ in $\ell^2$, and thus also pointwise. So $f_\phi = 0$.

This is basically an example of a somewhat paradoxical fact that's bitten me before. Let $X,Y$ be Banach spaces, and suppose $E \subset X$ is dense. It's well known that every bounded operator $T : E \to Y$ has a unique bounded extension $\tilde{T} : X \to Y$. But it's possible that $T$ is injective while $\tilde{T}$ is not. Indeed, we've just constructed an example, by letting $E = \mathcal{M}$, $X = \widetilde{\mathcal{M}}$, $Y = \ell^2$, and $T : E \to Y$ the inclusion map.

The error is in this line:

The standard argument shows that $\widetilde{\mathcal{M}}$ is an RKHS of functions on $X$.

In fact, this is not generally true. The completion $\widetilde{\mathcal{M}}$ may not be naturally identified with a space of functions on $X$.

The "obvious" way that one would try to prove this is as follows. Consider an element $\phi \in \widetilde{\mathcal{M}}$. Since $\mathcal{M}$ is dense in its completion, there is a sequence $\{f_n\} \subset \mathcal{M}$ such that $f_n \to \phi$ in $\widetilde{\mathcal{M}}$-norm, which is an extension of the $\mathcal{M}$-norm. In particular, $\{f_n\}$ is $\mathcal{M}$-norm Cauchy. Because of your inequality $|f(x)| \le \|f\|_{\mathcal{M}} \sqrt{K(x,x)}$ (*), we have that $\{f_n(x)\}$ is Cauchy in $\mathbb{R}$ for each $x$. So $\{f_n(x)\}$ converges to a number which we may call $f_\phi(x)$; that is, $f_n \to f_\phi$ pointwise. It is also easy to show that $f_{\phi}$ does not depend on the choice of sequence $f_n \to \phi$, so the linear map $\phi \mapsto f_\phi$ of $\widetilde{\mathcal{M}}$ into $\mathbb{R}^X$ is well defined. Let's call this map $T$.

So the "obvious" thing to do is to look at the image of $T$, which is of course a function space, and make it a Hilbert space by pushing forward the $\widetilde{\mathcal{M}}$-norm. If this works, then the same inequality (*) will show that the evaluation map is continuous in this norm, and we have ourselves an RKHS.

The problem is that $T$ might fail to be injective. In other words, we could have a nonzero $\phi$ for which $f_\phi = 0$. In that case, pushing forward the norm will not work; it will not be well defined on the image of $T$.

This is exactly what happens in the counterexample you discuss in the comments. As you suggest, consider $\ell^2$ as an RKHS on $X = \mathbb{N} = \{1,2,\dots\}$, with its usual orthonormal basis $\{e_n\}$, and let $v = \sum_n \frac{1}{n} e_n$. Let $H = \{v\}^\perp$ with the same $\ell^2$ inner product; being a closed subspace of $\ell^2$, $H$ is an RKHS.

Let $P : \ell^2 \to H$ be the orthogonal projection and let $g_n = P e_n$. The $g_n$ are linearly independent; if $0 = a_1 g_1 + \dots + a_n g_n = P(a_1 e_1 + \dots + a_n e_n)$ then $a_1 e_1 + \dots + a_n e_n$ is a scalar multiple of $v$, which is only possible if all $a_i$ are 0.

Following your construction, let $\mathcal{M} \subset H$ be the linear span of $\{g_n\}$ and let $\langle\cdot, \cdot\rangle_{\mathcal{M}}$ be the inner product on $\mathcal{M}$ which makes the $\{g_n\}$ orthonormal.

Set $h_m = \sum_{n=1}^m \frac{1}{n} g_n$. Clearly the sequence $\{h_m\}$ is Cauchy in $\mathcal{M}$, so it converges in the completion $\widetilde{\mathcal{M}}$ to some $\phi$. It is also clear that $\|\phi\|_\widetilde{\mathcal{M}}^2 = \frac{\pi^2}{6}$ so in particular $\phi \ne 0$.

But on the other hand, we have $h_m = \sum_{n=1}^m \frac{1}{n} P e_n = P\left(\sum_{n=1}^m \frac{1}{n} e_n\right)$. By continuity of $P$, we have $h_m \to Pv = 0$ in $\ell^2$, and thus also pointwise. So $f_\phi = 0$.

This is basically an example of a somewhat paradoxical fact that's bitten me before. Let $X,Y$ be Banach spaces, and suppose $E \subset X$ is dense. It's well known that every bounded operator $T : E \to Y$ has a unique bounded extension $\tilde{T} : X \to Y$. But it's possible that $T$ is injective while $\tilde{T}$ is not. Indeed, we've just constructed an example, by letting $E = \mathcal{M}$, $X = \widetilde{\mathcal{M}}$, $Y = \ell^2$, and $T : E \to Y$ the inclusion map.

The error is in this line:

The standard argument shows that $\widetilde{\mathcal{M}}$ is an RKHS of functions on $X$.

In fact, this is not generally true. The completion $\widetilde{\mathcal{M}}$ may not be naturally identified with a space of functions on $X$.

The "obvious" way that one would try to prove this is as follows. Consider an element $\phi \in \widetilde{\mathcal{M}}$. Since $\mathcal{M}$ is dense in its completion, there is a sequence $\{f_n\} \subset \mathcal{M}$ such that $f_n \to \phi$ in $\widetilde{\mathcal{M}}$-norm, which is an extension of the $\mathcal{M}$-norm. In particular, $\{f_n\}$ is $\mathcal{M}$-norm Cauchy. Because of your inequality $|f(x)| \le \|f\|_{\mathcal{M}} \sqrt{K(x,x)}$ (*), we have that $\{f_n(x)\}$ is Cauchy in $\mathbb{R}$ for each $x$. So $\{f_n(x)\}$ converges to a number which we may call $f_\phi(x)$; that is, $f_n \to f_\phi$ pointwise. It is also easy to show that $f_{\phi}$ does not depend on the choice of sequence $f_n \to \phi$, so the linear map $\phi \mapsto f_\phi$ of $\widetilde{\mathcal{M}}$ into $\mathbb{R}^X$ is well defined. Let's call this map $T$.

So the "obvious" thing to do is to look at the image of $T$, which is of course a function space, and make it a Hilbert space by pushing forward the $\widetilde{\mathcal{M}}$-norm. If this works, then the same inequality (*) will show that the evaluation map is continuous in this norm, and we have ourselves an RKHS.

The problem is that $T$ might fail to be injective. In other words, we could have a nonzero $\phi$ for which $f_\phi = 0$. In that case, pushing forward the norm will not work; it will not be well defined on the image of $T$.

This is exactly what happens in the counterexample you discuss in the comments. As you suggest, consider $\ell^2$ as an RKHS on $X = \mathbb{N} = \{1,2,\dots\}$, with its usual orthonormal basis $\{e_n\}$, and let $v = \sum_n \frac{1}{n} e_n$. Let $H = \{v\}^\perp$ with the same $\ell^2$ inner product; being a closed subspace of $\ell^2$, $H$ is an RKHS.

Let $P : \ell^2 \to H$ be the orthogonal projection and let $g_n = P e_n$. Following your construction, let $\mathcal{M} \subset H$ be the linear span of $\{g_n\}$ and let $\langle\cdot, \cdot\rangle_{\mathcal{M}}$ be the inner product on $\mathcal{M}$ which makes the $\{g_n\}$ orthonormal.

Set $h_m = \sum_{n=1}^m \frac{1}{n} g_n$. Clearly the sequence $\{h_m\}$ is Cauchy in $\mathcal{M}$, so it converges in the completion $\widetilde{\mathcal{M}}$ to some $\phi$. It is also clear that $\|\phi\|_\widetilde{\mathcal{M}}^2 = \frac{\pi^2}{6}$ so in particular $\phi \ne 0$.

But on the other hand, we have $h_m = \sum_{n=1}^m \frac{1}{n} P e_n = P\left(\sum_{n=1}^m \frac{1}{n} e_n\right)$. By continuity of $P$, we have $h_m \to Pv = 0$ in $\ell^2$, and thus also pointwise. So $f_\phi = 0$.

The error is in this line:

The standard argument shows that $\widetilde{\mathcal{M}}$ is an RKHS of functions on $X$.

In fact, this is not generally true. The completion $\widetilde{\mathcal{M}}$ may not be naturally identified with a space of functions on $X$.

The "obvious" way that one would try to prove this is as follows. Consider an element $\phi \in \widetilde{\mathcal{M}}$. Since $\mathcal{M}$ is dense in its completion, there is a sequence $\{f_n\} \subset \mathcal{M}$ such that $f_n \to \phi$ in $\widetilde{\mathcal{M}}$-norm, which is an extension of the $\mathcal{M}$-norm. In particular, $\{f_n\}$ is $\mathcal{M}$-norm Cauchy. Because of your inequality $|f(x)| \le \|f\|_{\mathcal{M}} \sqrt{K(x,x)}$ (*), we have that $\{f_n(x)\}$ is Cauchy in $\mathbb{R}$ for each $x$. So $\{f_n(x)\}$ converges to a number which we may call $f_\phi(x)$; that is, $f_n \to f_\phi$ pointwise. It is also easy to show that $f_{\phi}$ does not depend on the choice of sequence $f_n \to \phi$, so the linear map $\phi \mapsto f_\phi$ of $\widetilde{\mathcal{M}}$ into $\mathbb{R}^X$ is well defined. Let's call this map $T$.

So the "obvious" thing to do is to look at the image of $T$, which is of course a function space, and make it a Hilbert space by pushing forward the $\widetilde{\mathcal{M}}$-norm. If this works, then the same inequality (*) will show that the evaluation map is continuous in this norm, and we have ourselves an RKHS.

The problem is that $T$ might fail to be injective. In other words, we could have a nonzero $\phi$ for which $f_\phi = 0$. In that case, pushing forward the norm will not work; it will not be well defined on the image of $T$.

This is exactly what happens in the counterexample you discuss in the comments. As you suggest, consider $\ell^2$ as an RKHS on $X = \mathbb{N} = \{1,2,\dots\}$, with its usual orthonormal basis $\{e_n\}$, and let $v = \sum_n \frac{1}{n} e_n$. Let $H = \{v\}^\perp$ with the same $\ell^2$ inner product; being a closed subspace of $\ell^2$, $H$ is an RKHS.

Let $P : \ell^2 \to H$ be the orthogonal projection and let $g_n = P e_n$. Following your construction, let $\mathcal{M} \subset H$ be the linear span of $\{g_n\}$ and let $\langle\cdot, \cdot\rangle_{\mathcal{M}}$ be the inner product on $\mathcal{M}$ which makes the $\{g_n\}$ orthonormal.

Set $h_m = \sum_{n=1}^m \frac{1}{n} g_n$. Clearly the sequence $\{h_m\}$ is Cauchy in $\mathcal{M}$, so it converges in the completion $\widetilde{\mathcal{M}}$ to some $\phi$. It is also clear that $\|\phi\|_\widetilde{\mathcal{M}}^2 = \frac{\pi^2}{6}$ so in particular $\phi \ne 0$.

But on the other hand, we have $h_m = \sum_{n=1}^m \frac{1}{n} P e_n = P\left(\sum_{n=1}^m \frac{1}{n} e_n\right)$. By continuity of $P$, we have $h_m \to Pv = 0$ in $\ell^2$, and thus also pointwise. So $f_\phi = 0$.

This is basically an example of a somewhat paradoxical fact that's bitten me before. Let $X,Y$ be Banach spaces, and suppose $E \subset X$ is dense. It's well known that every bounded operator $T : E \to Y$ has a unique bounded extension $\tilde{T} : X \to Y$. But it's possible that $T$ is injective while $\tilde{T}$ is not. Indeed, we've just constructed an example, by letting $E = \mathcal{M}$, $X = \widetilde{\mathcal{M}}$, $Y = \ell^2$, and $T : E \to Y$ the inclusion map.