About covariance operators for probability distributions on a function space Feel free to restrict the function space to a Hilbert space or to a RKHS. Given a probability distribution on it when can we define a ``covariance operator" for it and when would it also have a well-defined notion of eigenfunctions for it? 
 A: Let $X$ be a random vector taking values in a
separable Hilbert space $H$ such that $E\|X\|^2<\infty$ and $E X=\mu$. Then the corresponding covariance operator $R\colon H\to H$ is defined by the formula 
\begin{equation*}
 Rx:=E\langle x,X-\mu\rangle (X-\mu)=E\overline{\langle X-\mu,x\rangle}(X-\mu) 
\end{equation*} 
for $x\in H$, so that for any vectors $x$ and $y$ in $H$, 
\begin{equation*}
 \langle Rx,y\rangle=E\overline{\langle X-\mu,x\rangle}\langle X-\mu,y\rangle,  
\end{equation*}
the covariance of the "$x$-coordinate" $\langle X,x\rangle$ and the "$y$-coordinate" $\langle X,y\rangle$ of the random vector $X$. 
A brief account on the compactness of the covariance operator and its spectral decomposition is given in Appendices E and F of the paper at \url{http://projecteuclid.org/euclid.ejs/1460463653}. 
If $X$ is a zero-mean square-integrable stochastic process on a closed bounded interval $I$, with a continuous covariance function, then, by the Karhunen--Loève theorem (\url{https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem#The_Wiener_process}), 
$X$ admits a representation as a series of the form $\sum_k Z_k e_k$, where the $e_k$'s form an orthonormal basis of $L^2(I)$ of eigenvectors of the covariance operator of $X$ and the $Z_k$'s are zero-mean random variables such that $EZ_jZ_k=\lambda_j\delta_{jk}$, where $\lambda_j$ is the eigenvalue corresponding to the eigenvector $e_j$, and $\delta_{jk}$ is the Kronecker symbol. 
The specific Karhunen--Loève decompositions for the Brownian motion (Wiener process) and the Brownian bridge can also be found at \url{https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem#The_Wiener_process}. 
