1
$\begingroup$

Feel free to restrict the function space to a Hilbert space or to a RKHS. Given a probability distribution on it when can we define a ``covariance operator" for it and when would it also have a well-defined notion of eigenfunctions for it?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Let $X$ be a random vector taking values in a separable Hilbert space $H$ such that $E\|X\|^2<\infty$ and $E X=\mu$. Then the corresponding covariance operator $R\colon H\to H$ is defined by the formula \begin{equation*} Rx:=E\langle x,X-\mu\rangle (X-\mu)=E\overline{\langle X-\mu,x\rangle}(X-\mu) \end{equation*} for $x\in H$, so that for any vectors $x$ and $y$ in $H$, \begin{equation*} \langle Rx,y\rangle=E\overline{\langle X-\mu,x\rangle}\langle X-\mu,y\rangle, \end{equation*} the covariance of the "$x$-coordinate" $\langle X,x\rangle$ and the "$y$-coordinate" $\langle X,y\rangle$ of the random vector $X$.

A brief account on the compactness of the covariance operator and its spectral decomposition is given in Appendices E and F of the paper at \url{http://projecteuclid.org/euclid.ejs/1460463653}.

If $X$ is a zero-mean square-integrable stochastic process on a closed bounded interval $I$, with a continuous covariance function, then, by the Karhunen--Loève theorem (\url{https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem#The_Wiener_process}), $X$ admits a representation as a series of the form $\sum_k Z_k e_k$, where the $e_k$'s form an orthonormal basis of $L^2(I)$ of eigenvectors of the covariance operator of $X$ and the $Z_k$'s are zero-mean random variables such that $EZ_jZ_k=\lambda_j\delta_{jk}$, where $\lambda_j$ is the eigenvalue corresponding to the eigenvector $e_j$, and $\delta_{jk}$ is the Kronecker symbol. The specific Karhunen--Loève decompositions for the Brownian motion (Wiener process) and the Brownian bridge can also be found at \url{https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem#The_Wiener_process}.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for this very helpful reference! Would you know if there are special cases of a distribution over an infinite dimensional function space where one explicitly knows these eigenfunctions of the covariance operator? $\endgroup$
    – gradstudent
    Commented Aug 6, 2016 at 22:12
  • $\begingroup$ I have added a paragraph on the Karhunen--Loève decomposition, including the specific examples of the Brownian motion and the Brownian bridge. $\endgroup$
    – Iosif Pinelis
    Commented Aug 7, 2016 at 4:34
  • $\begingroup$ Thanks! So this covariance operator is of the ``Wiener distribution" on the space of $\mathbb{R} \rightarrow \mathbb{R}$, right? Here the "stochastic process" is just an intermediary and not really the important point - am I right? And the $W_t$ whose spectral decomposition is given here, en.wikipedia.org/wiki/Wiener_process#Wiener_representation is really the value of a randomly sampled function using the the Wiener distribution. Right? $\endgroup$
    – gradstudent
    Commented Aug 11, 2016 at 18:09
  • $\begingroup$ Is there any analogous "Wiener distribution" and corresponding such spectral decomposition for the covariance matrix on a more general space of $\mathbb{R}^n \rightarrow \mathbb{R}^m$ functions? $\endgroup$
    – gradstudent
    Commented Aug 11, 2016 at 18:11
  • $\begingroup$ Any (say) continuous square-integrable random process $X=(X(t))_{t\in I}$ (such as the Wiener process) on a closed bounded interval $I$ can indeed be viewed as a random function in $L^2(I)$. Assume, for simplicity of writing, that $E X=0$ (as is the case with the Wiener process). Then the covariance operator $R$ of $X$ is the integral transform with kernel function $I^2\ni(t,s)\mapsto K(t,s):=EX(t)\overline{X(s)}$. That is, $(Rx)(t)=E\langle x,X\rangle X(t)=EX(t)\int_I\overline{X(s)}x(s)\,ds=\int_I K(t,s)x(s)\,ds$ for all $x\in L^2(I)$ and $t\in I$. $\endgroup$
    – Iosif Pinelis
    Commented Aug 11, 2016 at 19:27

You must log in to answer this question.