A geometric interpretation of independence?

Question

Consider the set of random variables with zero mean and finite second moment. This is a vector space, and $\langle X, Y \rangle = E[XY]$ is a valid inner product on it. Uncorrelated random variables correspond to orthogonal vectors in this space.

Questions:

(i) Does there exist a similar geometric interpretation for independent random variables in terms of this vector space?

(ii) A collection of jointly Gaussian random variables are uncorrelated if and only if they are independent. Is it possible to give a geometric interpretation for this?

Answer 1

There is a Hilbert space interpretation of independence, which follows from the interpretation of conditional expectation as an orthogonal projection, though it may be more complicated than you had in mind.

Say your underlying probability space is $(\Omega, \mathcal{F}, \mathbb{P})$, and write $L^2(\mathcal{F})$ for the Hilbert space of ($\mathcal{F}$-measurable) random variables with finite variance (with $\Omega$ and $\mathbb{P}$ understood). Denote by $\sigma(X)$ the $\sigma$-algebra generated by the random variable $X$. Now the conditional expectation $\mathbb{E}[X|Y]$ is the orthogonal projection in $L^2(\mathcal{F})$ of $X$ onto the subspace $L^2(\sigma(Y))$ of random variables which are $\sigma(Y)$-measurable. $X$ and $Y$ are independent if and only if $\mathbb{E}[f(X)|Y]=\mathbb{E}f(X)$ for every reasonable function $f$. The functions $f(X)$ span $L^2(\sigma(X))$.

So if I now define $L^2_0(\sigma(X))$ to be the mean 0, finite variance random variables which are $\sigma(X)$-measurable, I can say: $X$ and $Y$ are independent iff $L^2_0(\sigma(X))$ is orthogonal to $L^2(\sigma(Y))$ in $L^2(\mathcal{F})$.

Answer 2

(i) Not if I understood you correctly. A Hilbert space is so very symmetric that given that orthogonality is necessary but not sufficient for independence, there isn't anything else to look for. To be precise, any set of orthogonal vectors can be carried onto any other set of orthogonal vectors of the same cardinality by a unitary transformation. (Well, to be honest the dimensions of the orthogonal complement must match too, but that doesn't help.)

(ii) If you mean geometric as seen in the Hilbert space you talk about, no, since the answer to (i) is no.

I can't rule out the possibility that you might turn these “no” answers to “yes” answers by adding a suffictient amount of structure to the space, but I don't think that is what you were asking?

Answer 3

For question (i):

Independence of $X$ and $Y$ on this space is equivalent to the orthogonality of families of variables we can construct from $X$ and $Y$.

For each measurable sets $A,B \subset \mathbb R$, we can construct modified indicator variables $X_A,Y_B$ with mean $0$ and finite second moment so that the orthogonality of $X_A$ and $Y_B$ means that the events $X\in A$ and $Y \in B$ are independent events.

$X_A = f_{X,A}(X)$ where

$\begin{matrix} f_{X,A}(x) =& Prob(X \notin A), x \in A \\\ & -Prob(X \in A),x \notin A\end{matrix}$

and we define $Y_B = f_{Y,B}(Y)$ similarly.

$X$ and $Y$ don't have to be members of this space to use this. Even if $X$ does not have $0$ mean, or any mean, each variable $X_A$ does.

Answer 4

If you leave the realm of abstract probability spaces and focus on probability in Banach spaces, there's a lot of geometry to take advantage of. Here's an example.

Let $X$ be a Banach space, and let $\mathbb P$ be a Radon probability measure on $X$ such that continuous linear functionals are square-integrable (i.e. $\int_X |f(x)|^2 ~d\mathbb P(x) < \infty$ for all $f \in X^*$). For example, $X = C([0,1])$ with Wiener measure $\mathbb P$.

These are sufficient conditions for there to exist a mean $m \in X$ and covariance operator $K : X^* \to X$ such that $$\mathbb Ef = f(m) \qquad \mathrm{and} \qquad \mathbb E (fg) - f(m)g(m) = f(Kg)$$ for all $f, g \in X^*$. One can show that

$$\mathbb P \left( m + \overline{KX^*} \right) = 1.$$

Under these very general assumptions, the probability concentrates on the affine subspace generated by the mean and covariance.

_{Reference: Vakhania, Tarieladze and Chobanyan, Probability Distributions in Banach Spaces}

Answer 5

This is a pretty trivial observation, but in that space the criteria for independence is that $(p\circ X, q \circ Y) = (p \circ X,1)(1,q \circ Y)$ for all measurable indicator functions $p,q : \mathbb{R} \to \mathbb{R}$ (or to abstract just slightly from the underlying space, $pp = p$ and $qq = q$ where multiplication is pointwise). I can't see any analogies to geometry from this form, but it seems clear that there is a fundamental difference from the sort of hilbert space geometry that you mentioned, since we are quantifying over a whole class of external objects (the indicator functions).

Answer 6

This is not an answer, but I am not allowed to comment now. Your space is not an inner product space, since from $E[X X]=0$ it follows that $X=0$ holds only a.e. To make it a vector space, you have to consider the quotient space. Two random variables are in the same congruence class if and only if they are equal a.e. Then you can define your inner product on that quotient space. The inner product is independent of the representative.

Answer 7

Well I don't have the answer neither but I have always wondered using the orthogonality comparison in the case of Gaussian Hilbert spaces, if there could be some non-Euclidian geometry in which we could have a peculiar definition of orthogonality for which this notion would match the independence of random variables.

Regards