Is there a Karhunen-Loeve theorem for discrete-time process?

For example, let $\left\{X_i\right\}$ be a sequence of independent random variable which are uniformly distributed on the set $\{-1,1\}$. Define $$ S_t = \sum_{i=1}^t X_i, $$ with $t$ an integer between 1 and $n$. I would like to write $S$ as $$ S_t = \sum_{j=1}^t \xi_j e_j \tag{1} $$ where the coefficient $\xi_j$ are random and pairwise independent.
The vectors $\left\{e_j\right\}_{j=1}^n$ form an orthonormal basis of $\mathbb R^n$.
The representation (1) should minimzed the total square error.

I tried to adapt the proof of the Karhunen-Loeve and Mercer theorem.
My guess is that the Mercer kernel, or covariance function, should now be $$ K(s, t) = E\left(X_s X_t \right). $$

I'm wondering if this kind of result is either:

  • already available from the standard theory? if so, how?
    (by appropriately defining the space maybe)
  • already treated somewhere? if so where?
    (reproducing kernel Hilbert space theory, or a generalization of the Karhunen-Loeve expansion)

I seek the representation (1) because I want to discretize, or quantize, the process $S$.


There does not seem to be a problem. I assume that you have a random field

$$S_t: \{1,\dotsc, n\}\to \mathbb{Z}$$

The covariance kernel is $(t\leq \tau)$ $\newcommand{\bE}{\mathbb{E}}$

$$ \bE(S_tS_\tau)=\bE(S_t^2)= \sum_{i=1}^t \bE(X_i^2)=t=\min(t,\tau). $$

Now all you need to do is find the eigenvalues and eigenvectors of the $n\times n$ symmetric matrix $(\min(i,j))_{1\leq i,j\leq n}$.

Note: Have a look at section 1.4 of these notes by Zeitouni.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy