# Do we have Karhunen–Loève expansion for White Noise?

Let $W$ be a random process (my White Noise) on $[-1,1]$ such that:

1. $W(t)$ is a normal random variable with mean $0$ and standard deviation $1$ for all $t \in [-1,1]$
2. $E(W(t)W(s)) = 0$ for all $t, s \in [-1,1]$ with $t \neq s$.

The covariance function is $$K(t,s) = E(W(t)W(s)) = \begin{cases} 1 & \text{if } t = s \\ 0 & \text{otherwise} \end{cases}$$

Karhunen–Loève Theorem states that if the covariance function $K(t,s)$ is continuous there exists a Karhunen–Loève expansion. However, the above covariance function is NOT continuous.

Is there any "Karhunen–Loève"-like expansion in this case? More generally, do we have some kinds of orthogonal expansion if the covariance function is not continuous?

• such a random process doesn't exists, $W(t)$ cannot be independent of $W(t+\epsilon)$ for infinitesimal value of $\epsilon$, because there is no associated probability measure : try yourself, you'll get $\mathcal{P}(W(t_1:t_2) \in A) = 0$ for any set $A$ of functions on $[t_1,t_2]$. hence, you have to consider a discrete time white-noise, or the continuous time Wiener process which is the limit of a sequence of discrete brownian motions ($\sum_{n \le N}$ of white noise) with sample rate $\to \infty$ (and $\sigma^2 \to 0$) Mar 17 '16 at 18:55
• Indeed, what you have written down is not the right way to mathematically define white noise. Any application that needs white noise should typically interpret it as the derivative of Brownian motion: either as a distribution-valued process, or by integrating and understanding $W_t = dB_t$ as the "differential" in a stochastic integral. Mar 18 '16 at 2:24

The process you describe (independent normal r.v.'s at every point) is not white noise. So there are really two questions here.

1. Does the process you describe have a KL expansion? The answer is no. One (informal) reason is that this process requires uncountably many Gaussian random variables to be described, while by definition, a KL expansion only uses countably many of them. Another (related) reason is that this process cannot be realised as a Borel measure on any reasonable function space. (It can be realised as a measure on the space of all functions endowed with its product $\sigma$-algebra, but this is not a Borel measure on the space of all functions endowed with the product topology.)

2. Does white noise have a KL expansion? The answer is yes, provided that you are willing to work in spaces of distributions rather than functions. White noise $\xi$ is a random distribution with $\mathbf{E} \xi(s)\xi(t) = \delta(t-s)$ and can be realised as $\xi = \sum_{n=1}^\infty e_n\, \xi_n$ for a sequence of i.i.d. Gaussians $\xi_n$ and any orthonormal basis $e_n$ of $L^2([0,1])$. This sum doesn't converge pointwise of course, but it converges for example in every Sobolev space $H^s$ for $s < -1/2$.

• the simplest way to understand $\xi$ is by considering the random variables $\Xi_\varphi = \langle \xi,\varphi \rangle = \sum_n \xi_n \langle e_n, \varphi \rangle$ which are a well-behavied normal distributed random variables whenever $\sum_n |\langle e_n, \varphi \rangle| < \infty$ Mar 17 '16 at 21:48
• and I didn't get the $H^s$ part. did you mean that it converges as a (random) linear functional $H^s \to \mathbb{R}$ (more or less what I wrote) ? Mar 17 '16 at 21:56
• Does the $\delta(\cdot)$ in the second point mean Dirac delta in which "$\delta(0) = \infty$"? Mar 18 '16 at 1:31
• Is there a place to read intuitive simple Math which derives the Auto Correlation Function of White Noise (Continuous) and shows the reasoning behind the use of $\delta \left( t \right)$?
– Royi
Jun 16 '18 at 19:40

Not in the case of white noise. If you managed to represent $W_t = \sum e_i(t)Z_i$ then the $W_t$ must live on a countable generated sigma field, on which e.g. $L^2$ is seperable, whereas the collection $W_t$ exhibit an uncountable orthonormal set. However, the continuity is not particularly important. If you have expanded the kernel as $\sum \phi_i(t)\phi_i(s)$ it assures that $\sum \phi_i^2(t) < \infty$ so that $\sum Z_i \phi_i(s) < \infty$, but as long as you can do that you are in good shape. For example if $K(s,t) = 1, s, t < \frac 12$ or $s,t > \frac 12$ and 0 otherwise the process can be written $e_1(t) Z_1 + e_2(t)Z_2$ where the $Z_i$ are i.i.d. normal and $e_1(t) = 1_{(0,\frac 12)}(t)$, $e_2(t) = 1_{ (\frac 12,1)}(t)$

• what I commented is that with $E[W(t)W(t+\epsilon)] = 0$ for infinitesimal $\epsilon$, it is not even a random process Mar 17 '16 at 19:05