# Need help understanding Riesz representation theorem for Reproducing Kernel Hilbert Spaces

I'd like some help understanding any of the following proofs of Riesz representation theorem -- whichever is simpler -- or in fact any proof of the theorem.

BTW, I am quite intrigued that the mere existence of a bounded linear functional is sufficient for there to exist a reproducing kernel.

As some of you may already be aware, I am computer scientist trying teaching myself a bit of functional analysis in order to better appreciate the mathematical methods I use. As such simple explanations will be appreciated.

Thanks,

Olumide

-
You should probably be more specific about what you don't understand. –  Qiaochu Yuan Aug 24 '10 at 23:47
I agree with Qiaochu Yuan (and also it's not research-level). However, trying to guess which part is causing you the most difficulty, I think the most difficult bit (which is not given in both of the above proofs) is the "Hilbert space decomposition theorem" or "Projection lemma", which boils down to the fact that given a point $x$ and a closed, convex set $C$ in a Hilbert space, there exists a UNIQUE point $c \in C$ closest to $x$, i.e. minimising $\|x-c\|$. This is really the central core of the argument which makes everything else work; ... –  Zen Harper Aug 25 '10 at 2:36
...and although this result is "intuitive" (since our physical intuition is for Euclidean 3-space), the proof is not so obvious, needing Cauchy sequences and the Parallelogram Rule. –  Zen Harper Aug 25 '10 at 2:36
To be honest, I'm having difficulty understanding the first line of proof 1 i.e. $ker f \bigoplus (ker f)^\bot$. Why $ker f$? –  Olumide Aug 25 '10 at 4:31
If that is the case, I suggest you first consider the finite dimensional case (standard linear algebra). Zen Harper's comment is useful for tidying up the proof and ensuring that the reasonings in the finite dimensional setting can be carried over to the Hilbert space setting. The rest of the proof is just linear algebra. –  Willie Wong Aug 25 '10 at 13:26

This was initially meant to be a comment, but became too long. So, firstly, I'd warmly suggest you to adopt and follow a good book on the subject, rather than using on-line material: which is good as a reference for single proofs, but not often enough orgainzed into a whole theory. The basic material on Hilbert spaces is quite easy and intuitive, but needs several little results, that fit better in a book. You may like Rudin's exposition in Real and Complex analysis (chapt. 4), or Akhiezer & Glazman's Theory of linear operators in Hilbert space (chapt. 1), or Halmos' books Introduction to Hilbert Space and A Hilbert space problem book,... &c: there are of course several very good elementary books on the subject: the best is visiting a library and choose yours. That said, as to the Riesz duality theorem, you may follow this path:

• Two non-zero linear functionals on a vector space have the same kernel if and only if they are scalar multiple of each other. This is easy linear algebra.

• Then, in order to represent a bounded linear functional $f$ on a Hilbert space via scalar product with a vector $u$, that is $f=\phi_u$ where $\phi_u(x):=(x\cdot u)$, the key point is to find $v\in H$ such that $\ker f = \ker \phi_v$ (so the representing vector $u$ will be some scalar multiple of $v$). But $\ker\phi_v$ is by definition the orthogonal of $v$, so if such a vector $v$ exists, it has to be a generator of the line $(\ker f)^\perp$ because of the (nontrivial) duality relation $V=(V^\perp)^\perp$ for closed linear subspaces.

• From the above you are led to consider the orthogonal decomposition you mentioned, and easily conclude.

However, orthogonal decompositions and projectors are a topic which is important in its own, and deserve a separate little study (see Zen Harper's comment above). You may start from the concept of metric projection on a convex set C of a Hilbert space H (here the completeness and the uniform convexity of the Hilbert norm ensure existence and uniqueness of the point of C minimal distance from a given point of H). Particularizing the convex to be a linear closed space $V$ you'll see (here is the magic of the Hilbert structure) that the corresponding metric projector on $V$ is a bounded linear operator, and gives you the orthogonal decomposition $H=V\oplus V^{\perp}$, and the above mentioned relation).

-

This really isn't an answer. Like Pietro's, it's a comment that got out of hand.

I've been reading a number of books on and offline (thanks to Google books), and I now understand what the kernel of a linear operator is as well as the orthogonal projection theorem, but an understanding of the proof still eludes me. (By the way I've noted that almost all the proofs I've found are versions of each other.) Nevertheless, reading so much about the proof has shed some light on the nature of RKHS, such as:

• any linear evaluation function $f(x) = \; \lt x , x_0 \gt$ is an inner product ($x_0$ is the representer of the evaluation function)
• for each evaluation function there exists only one $x_0 \in H$
• $\parallel f \parallel \; = \; \parallel x_0 \parallel$

Furthermore, according to "Smoothing Spline ANOVA Models" Gu, Chong 2002 (page 27)

"For every $g$ in a Hilbert space $\mathcal{H}$, $L_gf \; = \; \lt g , f \gt$ defines a continuous linear functional $L_g$. Conversely, every continuous linear functional $L$ in $\mathcal{H}$ has a representation $Lf \; = \; \lt g_L , f \gt$ for some $g_L \in \mathcal{H}$, called the representer of the evaluation."

This statement demystifies RKHS by the assertion that: every linear evaluation functional is (merely) an inner product of the representer and an element of the RKHS, with the result that the Riesz representation is increasingly seems to to be a definition i.e. something to be accepted and not a result that must be derived.

-
"with the result that the Riesz representation is increasingly seems to be a definition" -- welcome to mathematics. Theorems are just (sometime complicated) ways of showing some assertion follows "by definition". Cf. Feynman's comment on mathematicians only proving trivial theorems. –  Willie Wong Aug 26 '10 at 12:49
Thanks Willie. So, if $L_gf \; = \; \lt g , f \gt$ is a definition, am I right in supposing that the only thing to be proved is the uniqueness of $g$? for every $f$. BTW, re the original assertion, am I right in thinking that one is free to refer to the $L_gf$ as an inner product if it: has conjugate symmetry, is linear in the first argument and positive definite in addition to being bounded? –  Olumide Aug 26 '10 at 15:43
I'm pleased to report that I now understand the first two parts of the proof (I found the explanation in "Principles of Functional Analysis" by Martin Schechter, to be very clear). However, the last part of the proof i.e. i.e. $\parallel f \parallel \; = \; \parallel x_0 \parallel$ still eludes me. To begin with, I don't understand the statement, $\parallel f \parallel = \stackrel{sup}{ \parallel x \parallel = 1} \mid f(x) \mid \le \parallel y \parallel$ –  Olumide Sep 2 '10 at 11:44
I meant $\parallel f \parallel = \underset{ \parallel x \parallel = 1}{sup} \mid f(x) \mid \le \parallel y \parallel$ –  Olumide Sep 2 '10 at 12:10
Arrgh!!! Your $L_g$ has $g$, $f$ the other way round to what I (and most of my other analyst friends) would do; be VERY CAREFUL if you're using complex numbers, because then swapping the order in the scalar product produces a complex conjugation, so your $L_g$ will NOT necessarily be linear; it might be conjugate-linear instead, if you use complex numbers. But it depends on your definitions! Some people, particularly physicists, like to define their scalar products the other way round to me. Anyway, the $\|f\|$ estimate is just the Cauchy-Schwarz inequality for the scalar product. –  Zen Harper Sep 10 '10 at 2:47