Let $\mathcal{H}$ be a separable Hilbert space. Let $v$ be a random variable taking values in $\mathcal{H}$ such that $P(v \perp h) < 1$ for all $h \in \mathcal{H}.$ Suppose we sample an infinite sequence $v_1, v_2, \ldots.$ Is it the case that, almost surely, the closed span of $v_1, v_2, \ldots$ is all of $\mathcal{H}?$

  • $\begingroup$ and if they are iid the probability of being in a closed hyperplane $(h)^\perp$ is $P(v_k\perp h, k=1,2,\dots)=0$ $\endgroup$ – Pietro Majer Apr 21 '19 at 17:07
  • 3
    $\begingroup$ @Pietro Majer: this is the probability that the vectors all lie in a given closed hyperplane. $\endgroup$ – Anthony Quas Apr 21 '19 at 17:10
  • 3
    $\begingroup$ Whoever voted to close this, I'm pretty sure you don't understand the question. This is subtle and interesting. $\endgroup$ – Anthony Quas Apr 21 '19 at 17:18
  • 1
    $\begingroup$ Maybe I'm missing something obvious, but is it even clear that the event you are interested in is measurable? $\endgroup$ – Jochen Glueck Apr 21 '19 at 17:21
  • 1
    $\begingroup$ @JochenGlueck: Yes it's measurable: Let $(y_n)$ be a dense sequence in $\mathcal H$. Then the event is: for all $m>0$, for all $n>0$, there exist $k>0$ and rational $t_1,\ldots,t_k$ such that $\|t_1v_1+\ldots+t_kv_k-y_n\|<1/m$. $\endgroup$ – Anthony Quas Apr 21 '19 at 19:18

(This may turn out to be a simplified version of J. E. Pascoe's answer).

The support of (the distribution of) $v$, that we denote by $\operatorname{supp} v$, is the set of vectors $h \in \mathcal{H}$ such that $P(v \in B(h, \varepsilon)) > 0$ for every $\varepsilon > 0$. We list some properties of this set.

  1. The set $\operatorname{supp} v$ is the complement of the union of all open sets $B$ such that $P(v \in B) = 0$. Thus, the support is a closed set.

  2. Since $\mathcal{H}$ is a separable metric space, it has a countable topological base $\mathcal{B}$, and $\operatorname{supp} v$ is the complement of the union of all $B \in \mathcal{B}$ such that $P(v \in B) = 0$. By countable additivity, it follows that $P(v \in \operatorname{supp} v) = 1$ (the support is a set of full measure).

  3. With probability one, the closure of the random set $V = \{v_1, v_2, \ldots\}$ contains $\operatorname{supp} v$. Indeed, let $\{h_1, h_2, \ldots\}$ be a countable, dense subset of $\operatorname{supp} v$. For every $i, n = 1, 2, \ldots$ we have $P(v \in B(h_i, \tfrac{1}{n})) > 0$, and thus, by Borel–Cantelli, $P(V \cap B(h_i, \tfrac{1}{n}) = \varnothing) = 0$. It follows that $h_i \in \overline{V}$ for every $i = 1, 2, \ldots$, and consequently $\operatorname{supp} v \subseteq \overline{V}$.

  4. For every $h \in \mathcal{H}$, we have $P(h \perp v) < 1$, and therefore $h$ is not orthogonal to $\operatorname{supp} v$. It follows that the closed span of $\operatorname{supp} v$ is $\mathcal{H}$.

It remains to note that the closed span of $V$ is the same as the closed span of the closure of $V$, which with probability one contains the closed span of $\operatorname{supp} v$, which we have shown to be equal to $\mathcal{H}$.

(Item 1 is valid for any topological space; items 2 and 3 work in an arbitrary separable metric space.)

  • $\begingroup$ It would perhaps be even clearer to say that for each fixed vector $h\in\textrm{supp}\; v$, we have $P(h\in \overline{V})=1$. (This shows that the assumption that $H$ is separable is used, and how you avoid the pitfall from Pietro's comment above.) $\endgroup$ – Christian Remling Apr 22 '19 at 19:53
  • $\begingroup$ @ChristianRemling: I edited my answer to emphasize where separability is essential. Let me know if anything is wrong. Thanks! $\endgroup$ – Mateusz Kwaśnicki Apr 23 '19 at 21:39
  • $\begingroup$ Looks good to me. I didn't mean to suggest that such an extensive edit was needed, of course, just wanted to draw attention to where your answer differs from Pietro's comment. $\endgroup$ – Christian Remling Apr 23 '19 at 23:13

Another Try

We say a $\mathcal{H}$-valued random variable $h$ is a random vector if $P(h \perp g)<1$ for all $g\in \mathcal{H}.$

If $h_1, h_2, \ldots$ is a sequence independent identically distributed of random vectors, then, almost surely, the closed span of the $h_i$ is equal to $\mathcal{H}.$

First we will need a lemma.

Lemma 1 Let $h$ be a random vector. There is a countable subset $A$ of $\mathcal{H}$ such that the closed span of the elements of $A$ is equal to $\mathcal{H}$ and for every point $a\in A,$ $P(h\in U)>0$ for any neighborhood $U$ of $a.$

Proof For any subset $A$ such that for every point $a\in A,$ $P(h\in U)>0$ for any neighborhood $U$ of $a,$ and the closed span of the elements of $A$ is not equal to $\mathcal{H},$ we will show that we can grow $A$ by a single element which is not in closed span of the elements of $A.$ We can only do this a countable number of times because the Hilbert space dimension of $\mathcal{H}$ is countable. (Otherwise, via Gram-Schmidt, we could construct an uncountable orthonormal set by transfinite induction.)

Choose $g$ such that $g \perp a$ for all $a\in A.$ Now, $P(h \perp g)<1.$ So there must be a point $b$ such that $P(h\in U) >0$ for every neighborhood of $b$ and $b$ is not perpendicular to $g,$ therefore, $b$ is not in the span of the elements of $A.$ QED

Suppose $h_1, h_2, \ldots$ is a sequence independent identically distributed of random vectors. Let $A$ be as in Lemma 1. Index $A$ a a sequence $a_n.$ Let $B_{m,n}$ be a ball of radius $1/m$ centered at $a_n$ Almost surely, the sequence $h_i$ must visit $B_{m,n}$ infinitely often, as $P(h_i\in B_{m,n})>0$. Therefore $A$ is a subset of the closure of the values of the sequence. (We have essentially the fact that a random function $f:\mathbb{N}\rightarrow \mathbb{N}^2$ is surjective with infinite multiplicity.)

  • $\begingroup$ The point is that you "keep going" by transfinite induction. The process must stop at some countable ordinal before $\omega_1$ as the space is countable dimensional. $\endgroup$ – J. E. Pascoe Apr 21 '19 at 19:59
  • $\begingroup$ That is for each $\alpha < \omega_1$ there would be $A_\alpha$ such that if $\alpha < \beta,$ $A_\alpha^\perp \cap A_\beta \neq \{0\}.$ $\endgroup$ – J. E. Pascoe Apr 21 '19 at 20:06
  • $\begingroup$ "Let $A$ be as in Lemma 1" ... "we may take $A$ to consist of only isolated points, as in a Polish space a countable closed set is equal to the closure of its isolated points." But Lemma 1 does not guarantee that $A$ is closed. $\endgroup$ – Iosif Pinelis Apr 21 '19 at 20:07
  • $\begingroup$ That does seem to be a gap @IosifPinelis . Ideas for closing it? $\endgroup$ – J. E. Pascoe Apr 21 '19 at 20:10
  • $\begingroup$ Could you elaborate a bit further on why the set $A$ in Lemma 1 is countable? Of course, there exist linearly independent sets in $H$ that are uncountable -- just choose your favourite Hamel basis of $H$. $\endgroup$ – Jochen Glueck Apr 21 '19 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.