Visualizing ANOVA Decomposition [closed]

Question

Let $f \in L^2[0,1]^d$ be a measurable function where $d \in \mathbb{N}$. For a given subset $u \subseteq D := \{1,2,\ldots,d\}$ consider the projections $P_u : L^2[0,1]^d \to L^2[0,1]^{|u|}$ given by $$ P_u f(x_u) = \int_{[0,1]^{d - |u|}} f(x) \, dx^{|u^c|}, $$ were $x_u = \{(x_{i_s})_{s=1}^{|u|} : i_s \in u\}$. The ANOVA decomposition of $f$ is given by $$ f(x) = \sum_{u \subseteq D} f_u(x_u) $$ where each $f_u$ is constructed recursively as $$ f_u(x_u) := P_u f(x_u) - \sum_{v \subsetneq u} f_v(x_v). $$ This decomposition is orthogonal in the $L^2[0,1]^d$ inner product; that is, $$ (f_u, f_g)_{L^2} = 0, \qquad u \neq v. $$ I'm trying to get a geometric intuition behind this decomposition, and would appreciate some insights. As an example let $d = 2$, so the ANOVA decomposition is given by $$ f(x_1, x_2) = f_{\emptyset} + f_{\{1\}}(x_1) + f_{\{2\}}(x_2) + f_{\{1,2\}}(x_1, x_2), $$ where \begin{align*} f_\emptyset & = \int_{[0,1]^2} f(x_1,x_2) \, dx_1 dx_2, \\ f_{\{1\}}(x_1) & = \int_{[0,1]} f(x_1,x_2) \, dx_2 - f_\emptyset, \\ f_{\{2\}}(x_2) & = \int_{[0,1]} f(x_1,x_2) \, dx_1 - f_\emptyset, \\ f_{\{1,2\}}(x_2) & = f(x_1,x_2) - f_{\{1\}}(x_1) - f_{\{2\}}(x_2) - f_\emptyset. \end{align*} The trouble I'm having is visualizing the orthogonality. It seems that since $f$ is a function of two variables, I should be able to project it onto its two coordinates, but the $f_{\{1,2\}}$ term is telling me there's another dimension.

Answer 1

Well, this is easy to visualize if you can visualize the four-dimensional space :). First, imagine two mutually orthogonal planes $l_0,l_1$ in $\mathbb{R}^3$, and denote $l_\emptyset:=l_0\cap l_1$. Clearly, each vector $v\in\mathbb{R}^3$ has a unique representation $$ v=v_0+v_1+v_\emptyset $$ such that $v_{0,1,\emptyset}\in l_{0,1,\emptyset}$ are mutually orthogonal.

Now, suppose the same takes place in $\mathbb{R}^4$. Then, in fact, every vector of the above form would lie in a 3D subspace spanned by $v_0$ and $v_1$, so the best you can achieve is the representation $$ v=v_{\{0,1\}}+v_0+v_1+v_\emptyset, $$ where $v_{\{0,1\}}$ is orthogonal to that subspace.

In your actual case, the relevant spaces are spaces of functions that depend on a subset of coordinates only. These are closed infinite-dimensional subspaces, mutually orthogonal (modulo their intersection), and such that $l_{u_1}\oplus l_{u_2}$ has infinite codimension in $l_{u_1\cup u_2}$.