# Does a random sequence of vectors span a Hilbert space?

Let $$\mathcal{H}$$ be a separable Hilbert space. Let $$v$$ be a random variable taking values in $$\mathcal{H}$$ such that $$P(v \perp h) < 1$$ for all $$h \in \mathcal{H}.$$ Suppose we sample an infinite sequence $$v_1, v_2, \ldots.$$ Is it the case that, almost surely, the closed span of $$v_1, v_2, \ldots$$ is all of $$\mathcal{H}?$$

• and if they are iid the probability of being in a closed hyperplane $(h)^\perp$ is $P(v_k\perp h, k=1,2,\dots)=0$ – Pietro Majer Apr 21 '19 at 17:07
• @Pietro Majer: this is the probability that the vectors all lie in a given closed hyperplane. – Anthony Quas Apr 21 '19 at 17:10
• Whoever voted to close this, I'm pretty sure you don't understand the question. This is subtle and interesting. – Anthony Quas Apr 21 '19 at 17:18
• Maybe I'm missing something obvious, but is it even clear that the event you are interested in is measurable? – Jochen Glueck Apr 21 '19 at 17:21
• @JochenGlueck: Yes it's measurable: Let $(y_n)$ be a dense sequence in $\mathcal H$. Then the event is: for all $m>0$, for all $n>0$, there exist $k>0$ and rational $t_1,\ldots,t_k$ such that $\|t_1v_1+\ldots+t_kv_k-y_n\|<1/m$. – Anthony Quas Apr 21 '19 at 19:18

(This may turn out to be a simplified version of J. E. Pascoe's answer).

The support of (the distribution of) $$v$$, that we denote by $$\operatorname{supp} v$$, is the set of vectors $$h \in \mathcal{H}$$ such that $$P(v \in B(h, \varepsilon)) > 0$$ for every $$\varepsilon > 0$$. We list some properties of this set.

1. The set $$\operatorname{supp} v$$ is the complement of the union of all open sets $$B$$ such that $$P(v \in B) = 0$$. Thus, the support is a closed set.

2. Since $$\mathcal{H}$$ is a separable metric space, it has a countable topological base $$\mathcal{B}$$, and $$\operatorname{supp} v$$ is the complement of the union of all $$B \in \mathcal{B}$$ such that $$P(v \in B) = 0$$. By countable additivity, it follows that $$P(v \in \operatorname{supp} v) = 1$$ (the support is a set of full measure).

3. With probability one, the closure of the random set $$V = \{v_1, v_2, \ldots\}$$ contains $$\operatorname{supp} v$$. Indeed, let $$\{h_1, h_2, \ldots\}$$ be a countable, dense subset of $$\operatorname{supp} v$$. For every $$i, n = 1, 2, \ldots$$ we have $$P(v \in B(h_i, \tfrac{1}{n})) > 0$$, and thus, by Borel–Cantelli, $$P(V \cap B(h_i, \tfrac{1}{n}) = \varnothing) = 0$$. It follows that $$h_i \in \overline{V}$$ for every $$i = 1, 2, \ldots$$, and consequently $$\operatorname{supp} v \subseteq \overline{V}$$.

4. For every $$h \in \mathcal{H}$$, we have $$P(h \perp v) < 1$$, and therefore $$h$$ is not orthogonal to $$\operatorname{supp} v$$. It follows that the closed span of $$\operatorname{supp} v$$ is $$\mathcal{H}$$.

It remains to note that the closed span of $$V$$ is the same as the closed span of the closure of $$V$$, which with probability one contains the closed span of $$\operatorname{supp} v$$, which we have shown to be equal to $$\mathcal{H}$$.

(Item 1 is valid for any topological space; items 2 and 3 work in an arbitrary separable metric space.)

• It would perhaps be even clearer to say that for each fixed vector $h\in\textrm{supp}\; v$, we have $P(h\in \overline{V})=1$. (This shows that the assumption that $H$ is separable is used, and how you avoid the pitfall from Pietro's comment above.) – Christian Remling Apr 22 '19 at 19:53
• @ChristianRemling: I edited my answer to emphasize where separability is essential. Let me know if anything is wrong. Thanks! – Mateusz Kwaśnicki Apr 23 '19 at 21:39
• Looks good to me. I didn't mean to suggest that such an extensive edit was needed, of course, just wanted to draw attention to where your answer differs from Pietro's comment. – Christian Remling Apr 23 '19 at 23:13

Another Try

We say a $$\mathcal{H}$$-valued random variable $$h$$ is a random vector if $$P(h \perp g)<1$$ for all $$g\in \mathcal{H}.$$

If $$h_1, h_2, \ldots$$ is a sequence independent identically distributed of random vectors, then, almost surely, the closed span of the $$h_i$$ is equal to $$\mathcal{H}.$$

First we will need a lemma.

Lemma 1 Let $$h$$ be a random vector. There is a countable subset $$A$$ of $$\mathcal{H}$$ such that the closed span of the elements of $$A$$ is equal to $$\mathcal{H}$$ and for every point $$a\in A,$$ $$P(h\in U)>0$$ for any neighborhood $$U$$ of $$a.$$

Proof For any subset $$A$$ such that for every point $$a\in A,$$ $$P(h\in U)>0$$ for any neighborhood $$U$$ of $$a,$$ and the closed span of the elements of $$A$$ is not equal to $$\mathcal{H},$$ we will show that we can grow $$A$$ by a single element which is not in closed span of the elements of $$A.$$ We can only do this a countable number of times because the Hilbert space dimension of $$\mathcal{H}$$ is countable. (Otherwise, via Gram-Schmidt, we could construct an uncountable orthonormal set by transfinite induction.)

Choose $$g$$ such that $$g \perp a$$ for all $$a\in A.$$ Now, $$P(h \perp g)<1.$$ So there must be a point $$b$$ such that $$P(h\in U) >0$$ for every neighborhood of $$b$$ and $$b$$ is not perpendicular to $$g,$$ therefore, $$b$$ is not in the span of the elements of $$A.$$ QED

Suppose $$h_1, h_2, \ldots$$ is a sequence independent identically distributed of random vectors. Let $$A$$ be as in Lemma 1. Index $$A$$ a a sequence $$a_n.$$ Let $$B_{m,n}$$ be a ball of radius $$1/m$$ centered at $$a_n$$ Almost surely, the sequence $$h_i$$ must visit $$B_{m,n}$$ infinitely often, as $$P(h_i\in B_{m,n})>0$$. Therefore $$A$$ is a subset of the closure of the values of the sequence. (We have essentially the fact that a random function $$f:\mathbb{N}\rightarrow \mathbb{N}^2$$ is surjective with infinite multiplicity.)

• The point is that you "keep going" by transfinite induction. The process must stop at some countable ordinal before $\omega_1$ as the space is countable dimensional. – J. E. Pascoe Apr 21 '19 at 19:59
• That is for each $\alpha < \omega_1$ there would be $A_\alpha$ such that if $\alpha < \beta,$ $A_\alpha^\perp \cap A_\beta \neq \{0\}.$ – J. E. Pascoe Apr 21 '19 at 20:06
• "Let $A$ be as in Lemma 1" ... "we may take $A$ to consist of only isolated points, as in a Polish space a countable closed set is equal to the closure of its isolated points." But Lemma 1 does not guarantee that $A$ is closed. – Iosif Pinelis Apr 21 '19 at 20:07
• That does seem to be a gap @IosifPinelis . Ideas for closing it? – J. E. Pascoe Apr 21 '19 at 20:10
• Could you elaborate a bit further on why the set $A$ in Lemma 1 is countable? Of course, there exist linearly independent sets in $H$ that are uncountable -- just choose your favourite Hamel basis of $H$. – Jochen Glueck Apr 21 '19 at 20:18