# Gaussian processes, sample paths and associated Hilbert space.

Given a Gaussian process on some topological space $T$, with a continuous covariance kernel $C(\cdot,\cdot)\colon T\times T\to R$, we can associate a Hilbert space, which is the reproducing kernel Hilbert space of real-valued functions on $T$, with $C$ as kernel function. This contruction is given in, for instance, R J Adler & J E Taylor: "Random Fields and Geometry", and surely a lot of other places. We can suppose the topological space $T$ is separable.

A very rapid review of the construction is: Define an inner product space $H_0$ as consisting of all real-valued functions on $T$ of the form $f(x) = \sum_{i=1}^n a_i C(x_i,x)$, for real numbers $a_i$ and points in $T$, $x_i$. We can define an inner product on $H_0$ by $\left\langle\sum a_i C(x_i,\cdot), \sum b_j C(y_j,\cdot)\right\rangle = \sum \sum a_i b_j C(x_i,y_j)$.

Then the reproducing kernel Hilbert space associated with our gaussian process is the completion $H$ of $H_0$.

Now, this strongly suggests (to be usefull, and by the Karhunen-Loéve theorem, which is based on this construction) that sample paths of our Gaussian process belongs to H with probability 1. This must be proved somewhere, but where? Anybody knows a reference?

• I guess you need $T$ to be seperable, right? Otherwise you can't define a process by specifying its finite dimenstional distributions. Mar 27, 2011 at 18:55

The question of continuity of a Gaussian process is a rich one with a lot of theory. Let $T$ be a compact index set, and suppose that $X_t$ is a mean-zero Gaussian process with covariance function $c(t,s)$. The continuity properties of the process $X_t$ are entirely determined by the covariance function.

One very simple condition uses the Kolmogorov continuity theorem. Let $d \ge 1$, and suppose that the index set $T$ is a compact subset of $\mathbb R^d$. Suppose that the covariance function $c$ satisfies $$c(t,t) - 2c(t,s) + c(s,s) \le C|t - s|^{d + \beta},$$ for some positive constants $C$ and $\beta$. Then the $d$-dimensional Kolmogorov theorem implies that, with probability one, there exists a continuous version of $X_t$ on $T$.

This is far from the most general sufficient condition for a Gaussian process to be continuous. Since you have a copy of Adler & Taylor handy, take a look at Section 1.3, Boundedness and Continuity.

• Maybe a naive question, but how does continuity of the sample paths relate to whether the sample paths are in the RKHS of the kernel? As far as I know, for most cases the sample paths are in the RKHS with probability 0, yet under very mild assumptions on the kernel the sample paths will be continuous with probability 1. Just wondering if there is something here I am not seeing. Thanks! Jan 2 at 16:41

No, it is not true for simple examples such as standard Brownian motion or a sequence of independent random variables.

1. Suppose $W$ is a standard Brownian motion on the interval $[0,T]$. The covariance kernel is $C(s,t)=\mathbb{E}[W_sW_t]=\min(s,t)$. Then, the associated reproducing kernel Hilbert space, $H$, is the set of absolutely continuous functions $f\colon[0,T]\to\mathbb{R}$ with $f(0)=0$ and $\int_0^T\left(\frac{df(t)}{dt}\right)^2\,dt < \infty$. However, sample paths of Brownian motion are nowhere differentiable with probability one, so have zero probability of being in $H$.
2. Suppose $X_1,X_2,\ldots$ is a sequence of independent standard normal random variables. Then, $T=\mathbb{N}$ and the covariance kernel is $C(m,n)=\mathbb{E}[X_mX_n]=1_{\{m=n\}}$. The reproducing kernel Hilbert space, $H$, is just $\ell^2(\mathbb{N})$. However, by the strong law of large numbers, $\sum_{m\le n}X_m^2$ grows at rate $n$, and $\sum_nX_n^2$ is almost surely infinite so, again, the process has paths with zero probability of being in $H$.

However, even though the Gaussian process ($X$, say) does not have sample paths lying in the Hilbert space $H$, what we can say is that it is described by a cylindrical measure on $H$. This means that it has consistently defined projections into the finite dimensional subspaces of $H$ or, equivalently, that we can consistently define the values of the "inner product" $\langle f, X\rangle$ for $f\in H$. In the first example above, where $X$ is a Brownian motion, we would have \begin{align} \langle f,X\rangle &= \int_0^T\frac{\partial f(t)}{\partial t}\frac{\partial X(t)}{\partial t}\,dt\\ &=\int_0^T\frac{\partial f(t)}{\partial t}\,dX(t) \end{align} The first expression on the right hand side does not make sense as $X$ is nowhere differentiable but, reinterpreting it as a stochastic integral with respect to $dX$, we get a meaningful expression. Similarly in the second example above, where $X_n$ is an IID sequence of standard normal variables, we can write $$\langle f,X\rangle = \sum_{n=1}^\infty f(n)X(n).$$ Even though $X$ is not in $H=\ell^2(\mathbb{N})$, this sum converges (both in the $L^p$ norms for $1\le p < \infty$ and almost surely).

In fact, $X$ has the canonical Gaussian measure on $H$ and, as mentioned in the Wikipedia link, it is never a true measure when $H$ is infinite dimensional. I also answered another question concerning cylinder measures and the canonical Gaussian distribution (link) a while ago which could be of relevance to this question.

I came across the same question and found this thread. It doesn't look like a decisive conclusion was reached.

The reference you were looking for is this paper, particularly Theorem 7.4. If I understand it correctly, it says the sample path is in any slighter "larger" RKHS with probability 1. Hope this is still helpful.

If the RKHS $$\mathcal{H}_C$$ associated to the kernel of your GP is infinite dimensional, then sample paths are in $$\mathcal{H}_C$$ with probability 0.