Exchangeability and conditional expectation

Question

The following problem arises in my research of interacting particle systems:

Suppose that $X_1, \ldots, X_n$ are exchangeable real-valued random variables. Let $f,g: \mathbb{R} \to \mathbb{R}$ be Borel measurable functions. For $i \neq j$, is it true that

$$ \mathbb{E} \Big[ f \Big( \mathbb{E} \big[ g(X_i) \Big| X_j \big] \Big) \Big] = \mathbb{E} \Big[ f \Big( \mathbb{E} \big[ g(X_j) \Big| X_i \big] \Big) \Big] ?$$

This result would hold, by the definition of exchangeability, IF we can show that for any random variables $X$ and $Y$,$$\mathbb{E} \big[ g(X) \Big| Y \big] = h(X, Y),$$ for some Borel measurable function $h : \mathbb{R}^2 \to \mathbb{R}$. But is this statement true? Most standard texts only prove that $$\mathbb{E} \big[ g(X) \Big| Y \big] = h^{(X)}( Y),$$ for some Borel measurable function $h^{(X)} : \mathbb{R} \to \mathbb{R}$ that depends on $X$.

Answer 1

Let's be careful with quantifiers. I am reading your second statement as "For every $g$ there exists $h$ such that for all $X,Y$ we have $\mathbb{E}[g(X) \mid Y] = h(X,Y)$". It's not true. Let's just take $g(x) = x$. By considering $X,Y$ where $X=Y$ is a coin flip, we see that we must have $h(0,0) = 0$. By taking $X,Y$ to be independent coin flips we see that we must have $h(0,0) = \frac{1}{2}$. Contradiction.

The claim you attribute to "standard texts" seems to be "for every $g$ and every $X$ there exists $h$ such that for every $Y$ we have $\mathbb{E}[g(X) \mid Y] = h(Y)$." That's also not true. Considering the same examples as above would give $h(0) = 0$ and $h(0) = \frac{1}{2}$.

What is true is the following, stated carefully:

For every probability measure $\mu$ on $\mathbb{R}^2$ and every $g$, there exists $h$ such that for every pair of random variables $(X,Y)$ whose joint distribution is $\mu$, we have $\mathbb{E}[g(X) \mid Y] = h(Y)$ a.s.

Or equivalently:

For every pair of random variables $(X,Y)$ and every $g$, there exists $h$ such that $\mathbb{E}[g(X) \mid Y] = h(Y)$ a.s. Moreover, if $(X', Y')$ has the same joint distribution as $(X,Y)$, then $\mathbb{E}[g(X') \mid Y'] = h(Y')$ a.s.

I wrote down the proof here. (Just replace $X$ with $g(X)$.)

This gives your desired highlighted statement, since by exchangeability $(X_i, X_j)$ has the same joint distribution as $(X_j, X_i)$. So the desired equation reads $\mathbb{E}[f(h(X_j))] = \mathbb{E}[f(h(X_i))]$ which is true since $X_i, X_j$ are identically distributed.

Answer 2

To actually compute $$\mathbb{E} \Big[ f \Big( \mathbb{E} \big[ g(X_i) \Big| X_j \big] \Big) \Big]$$ only the Law of $(X_i,X_j)$ is needed, which happens, by exchangeability, to be the same as that of $(X_j,X_i)$. In summary, yes, your initial question is true.