Let $\mathcal{H}$ be a separable Hilbert space. Let $v$ be a random variable taking values in $\mathcal{H}$ such that $P(v \perp h) < 1$ for all $h \in \mathcal{H}.$ Suppose we sample an infinite sequence $v_1, v_2, \ldots.$ Is it the case that, almost surely, the closed span of $v_1, v_2, \ldots$ is all of $\mathcal{H}?$

$\begingroup$ and if they are iid the probability of being in a closed hyperplane $(h)^\perp$ is $P(v_k\perp h, k=1,2,\dots)=0$ $\endgroup$ – Pietro Majer Apr 21 at 17:07

3$\begingroup$ @Pietro Majer: this is the probability that the vectors all lie in a given closed hyperplane. $\endgroup$ – Anthony Quas Apr 21 at 17:10

3$\begingroup$ Whoever voted to close this, I'm pretty sure you don't understand the question. This is subtle and interesting. $\endgroup$ – Anthony Quas Apr 21 at 17:18

1$\begingroup$ Maybe I'm missing something obvious, but is it even clear that the event you are interested in is measurable? $\endgroup$ – Jochen Glueck Apr 21 at 17:21

1$\begingroup$ @JochenGlueck: Yes it's measurable: Let $(y_n)$ be a dense sequence in $\mathcal H$. Then the event is: for all $m>0$, for all $n>0$, there exist $k>0$ and rational $t_1,\ldots,t_k$ such that $\t_1v_1+\ldots+t_kv_ky_n\<1/m$. $\endgroup$ – Anthony Quas Apr 21 at 19:18
(This may turn out to be a simplified version of J. E. Pascoe's answer).
The support of (the distribution of) $v$, that we denote by $\operatorname{supp} v$, is the set of vectors $h \in \mathcal{H}$ such that $P(v \in B(h, \varepsilon)) > 0$ for every $\varepsilon > 0$. We list some properties of this set.
The set $\operatorname{supp} v$ is the complement of the union of all open sets $B$ such that $P(v \in B) = 0$. Thus, the support is a closed set.
Since $\mathcal{H}$ is a separable metric space, it has a countable topological base $\mathcal{B}$, and $\operatorname{supp} v$ is the complement of the union of all $B \in \mathcal{B}$ such that $P(v \in B) = 0$. By countable additivity, it follows that $P(v \in \operatorname{supp} v) = 1$ (the support is a set of full measure).
With probability one, the closure of the random set $V = \{v_1, v_2, \ldots\}$ contains $\operatorname{supp} v$. Indeed, let $\{h_1, h_2, \ldots\}$ be a countable, dense subset of $\operatorname{supp} v$. For every $i, n = 1, 2, \ldots$ we have $P(v \in B(h_i, \tfrac{1}{n})) > 0$, and thus, by Borel–Cantelli, $P(V \cap B(h_i, \tfrac{1}{n}) = \varnothing) = 0$. It follows that $h_i \in \overline{V}$ for every $i = 1, 2, \ldots$, and consequently $\operatorname{supp} v \subseteq \overline{V}$.
For every $h \in \mathcal{H}$, we have $P(h \perp v) < 1$, and therefore $h$ is not orthogonal to $\operatorname{supp} v$. It follows that the closed span of $\operatorname{supp} v$ is $\mathcal{H}$.
It remains to note that the closed span of $V$ is the same as the closed span of the closure of $V$, which with probability one contains the closed span of $\operatorname{supp} v$, which we have shown to be equal to $\mathcal{H}$.
(Item 1 is valid for any topological space; items 2 and 3 work in an arbitrary separable metric space.)

$\begingroup$ It would perhaps be even clearer to say that for each fixed vector $h\in\textrm{supp}\; v$, we have $P(h\in \overline{V})=1$. (This shows that the assumption that $H$ is separable is used, and how you avoid the pitfall from Pietro's comment above.) $\endgroup$ – Christian Remling Apr 22 at 19:53

$\begingroup$ @ChristianRemling: I edited my answer to emphasize where separability is essential. Let me know if anything is wrong. Thanks! $\endgroup$ – Mateusz Kwaśnicki Apr 23 at 21:39

$\begingroup$ Looks good to me. I didn't mean to suggest that such an extensive edit was needed, of course, just wanted to draw attention to where your answer differs from Pietro's comment. $\endgroup$ – Christian Remling Apr 23 at 23:13
Another Try
We say a $\mathcal{H}$valued random variable $h$ is a random vector if $P(h \perp g)<1$ for all $g\in \mathcal{H}.$
If $h_1, h_2, \ldots$ is a sequence independent identically distributed of random vectors, then, almost surely, the closed span of the $h_i$ is equal to $\mathcal{H}.$
First we will need a lemma.
Lemma 1 Let $h$ be a random vector. There is a countable subset $A$ of $\mathcal{H}$ such that the closed span of the elements of $A$ is equal to $\mathcal{H}$ and for every point $a\in A,$ $P(h\in U)>0$ for any neighborhood $U$ of $a.$
Proof For any subset $A$ such that for every point $a\in A,$ $P(h\in U)>0$ for any neighborhood $U$ of $a,$ and the closed span of the elements of $A$ is not equal to $\mathcal{H},$ we will show that we can grow $A$ by a single element which is not in closed span of the elements of $A.$ We can only do this a countable number of times because the Hilbert space dimension of $\mathcal{H}$ is countable. (Otherwise, via GramSchmidt, we could construct an uncountable orthonormal set by transfinite induction.)
Choose $g$ such that $g \perp a$ for all $a\in A.$ Now, $P(h \perp g)<1.$ So there must be a point $b$ such that $P(h\in U) >0$ for every neighborhood of $b$ and $b$ is not perpendicular to $g,$ therefore, $b$ is not in the span of the elements of $A.$ QED
Suppose $h_1, h_2, \ldots$ is a sequence independent identically distributed of random vectors. Let $A$ be as in Lemma 1. Index $A$ a a sequence $a_n.$ Let $B_{m,n}$ be a ball of radius $1/m$ centered at $a_n$ Almost surely, the sequence $h_i$ must visit $B_{m,n}$ infinitely often, as $P(h_i\in B_{m,n})>0$. Therefore $A$ is a subset of the closure of the values of the sequence. (We have essentially the fact that a random function $f:\mathbb{N}\rightarrow \mathbb{N}^2$ is surjective with infinite multiplicity.)

$\begingroup$ The point is that you "keep going" by transfinite induction. The process must stop at some countable ordinal before $\omega_1$ as the space is countable dimensional. $\endgroup$ – J. E. Pascoe Apr 21 at 19:59

$\begingroup$ That is for each $\alpha < \omega_1$ there would be $A_\alpha$ such that if $\alpha < \beta,$ $A_\alpha^\perp \cap A_\beta \neq \{0\}.$ $\endgroup$ – J. E. Pascoe Apr 21 at 20:06

$\begingroup$ "Let $A$ be as in Lemma 1" ... "we may take $A$ to consist of only isolated points, as in a Polish space a countable closed set is equal to the closure of its isolated points." But Lemma 1 does not guarantee that $A$ is closed. $\endgroup$ – Iosif Pinelis Apr 21 at 20:07

$\begingroup$ That does seem to be a gap @IosifPinelis . Ideas for closing it? $\endgroup$ – J. E. Pascoe Apr 21 at 20:10

$\begingroup$ Could you elaborate a bit further on why the set $A$ in Lemma 1 is countable? Of course, there exist linearly independent sets in $H$ that are uncountable  just choose your favourite Hamel basis of $H$. $\endgroup$ – Jochen Glueck Apr 21 at 20:18