-1
$\begingroup$

Suppose I have a random variable $x$ and a set of conditional distributions on $x$. Here is an example where the conditionals are nested: $$q_1 := E(x|y_1), \quad q_2 := E(x|y_1,y_2),\quad q_3 := E(x|y_1,y_2,y_3)$$ What I'd like to find is an orthogonal decomposition of $x$. For the example above, we have: $$ x = q_1 \oplus (q_2-q_1) \oplus (q_3-q_2) \oplus (x-q_3) $$ Is there a way to perform such an orthogonal decomposition when the conditionals are not nested? For example, what if we had: $$q_1 := E(x|y_1),\quad q_2 := E(x|y_1,y_2),\quad q_3 := E(x|y_1,y_3)$$

$\endgroup$
0
$\begingroup$

What about the standard Gram-Schmidt procedure?

Given any collection of expectations $q_1, q_2, \ldots, q_n$, we can write: $$ X = q_1 + (q_2-q_1) + \cdots + (q_n-q_{n-1}) + (X-q_n) $$ Now define $v_1=q_1$, $v_2 = q_2-q_1$, ..., $v_n=q_n-q_{n-1}$, $v_{n+1} = X-q_n$, so $X = \sum_{i=1}^{n+1} v_i$. The inner product between $v_i$ and $v_j$ is $E[v_iv_j]$, and $||v_i||^2 = E[v_i^2]$. So by Gram-Schmidt we can define: \begin{align} u_1 &= \frac{v_1}{||v_1||} \\ u_2 &= \frac{v_2 - E[v_2u_1]u_1}{||numerator||} \\ u_3 &= \frac{v_3 - E[v_3u_1]u_1 - E[v_3u_2]u_2}{||numerator||} \end{align} and so on, assuming at each step that the denominator is nonzero (else, we throw that particular vector away as it is a linear combination of the others). Then $E[u_iu_j]=0$ for all $i \neq j$, and $X = \sum_{i=1}^{n+1} E[Xu_i]u_i$.


For your example: $q_1=E[X|Y_1], q_2 = E[X|Y_1,Y_2], q_3 = E[X|Y_1,Y_3]$, so: \begin{align} v_1 &= E[X|Y_1]\\ v_2 &= E[X|Y_1,Y_2] - E[X|Y_1]\\ v_3 &= E[X|Y_1,Y_3] - E[X|Y_1,Y_2]\\ v_4 &= X - E[X|Y_1,Y_3] \end{align} and: \begin{align} u_1 &= \frac{E[X|Y_1]}{\sqrt{E[v_1^2]}} \\ u_2 &= \frac{v_2 - E[v_2u_1]u_1}{||numerator||} = \frac{v_2}{\sqrt{E[v_2^2]}}\\ u_3 &= \frac{v_3 - E[v_3u_1]u_1 - E[v_3u_2]u_2}{||numerator||} = \frac{v_3-E[v_3u_2]u_2}{||numerator||}\\ u_4 &= \frac{v_4 - E[v_4u_1]u_1 - E[v_4u_2]u_2-E[v_4u_3]u_3}{||numerator||} = \frac{v_4 - E[v_4u_2]u_2 - E[v_4u_3]u_3}{||numerator||} \end{align} and so $\{u_1, \ldots, u_4\}$ are orthonormal and $X = \sum_{i=1}^4 E[Xu_i]u_i$.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.