Finding joint probability from double marginals

Question

Consider three probability distributions in the form $p_1(y,z),p_2(x,z),p_3(x,y)$.

When does a global joint probability $p(x,y,z)$ (possibly not unique) exist?

The first compatibility condition to check is of course that the first order marginals check out: $p_2(x)=p_3(x)$, and so on. Is this the only condition, is it necessary and sufficient? Where can I find it?

Thanks!

PS. I would also be curious about what happens in any order, not just 2 and three...if it's possible!

Note: cross-posting from MSE.

Answer 1

No, the condition on the marginals is not sufficient. Consider for example three 0-1 valued random variables with bivariate marginals $p_i(s,t) = 0.5(1-\delta(s,t))$ for all $i, s, t$. The necessary compatibility condition on the univariate marginals is satisfied, but the bivariate marginals say that the three random variables $X$, $Y$, and $Z$ are pairwise distinct with probability one, contradicting the fact that they can only take two values. So there is no distribution $p(x,y,z)$ with these bivariate marginals.

As for sufficient conditions, in general there's nothing interesting you can say, but in some related cases there is. For example, the general theory of graphical models says that if you make a graph whose nodes are variables and whose edges are bivariate marginals you'd like to specify, then if this graph is a tree, the necessary compatibility condition is also sufficient, assuming some very weak regularity on the measures involved (anything on finite domains or absolutely continuous w.r.t Lebesgue measure is fine).

For more about the regularity issue, see "A Conditional Product Measure Theorem" by Swart. That paper includes a very clever example of two bivariate measures $p_{X,Y}$ and $p_{X,Z}$ whose $X$-marginals agree but which have no common extension.

Answer 2

The problem is one of linear programming. Indeed, suppose that $X$, $Y$, and $Z$ are finite sets. Then the problem is whether there exist nonnegative real numbers $p(x,y,z)$ such that $\sum_{x\in X}p(x,y,z)=p_1(y,z)$ for all $(y,z)\in Y\times Z$, $\sum_{y\in Y}p(x,y,z)=p_2(x,z)$ for all $(x,z)\in X\times Z$, and $\sum_{z\in Z}p(x,y,z)=p_3(x,y)$ for all $(x,y)\in X\times Y$. More generally, this may be a problem of infinite-dimensional linear programming.

The fundamental paper "The Existence of Probability Measures with Given Marginals" by Strassen (1965) in The Annals of Mathematical Statistics deals with the existence of a probability measure $\mu$ on the product space $X\times Y$ given the marginals $\mu\pi_X^{-1}$ and $\mu\pi_Y^{-1}$ (where $\pi_X$ and $\pi_Y$ are the projections from $X\times Y$ to $X$ and $Y$, respectively) plus further affine restrictions on $\mu$. At least in principle, the case of the product of more than two spaces should be reducible to Strassen's setting.

For instance, suppose that, given three spaces $X$, $Y$, and $Z$ with probability measures $\mu_1$, $\mu_2$, and $\mu_3$ over $Y\times Z$, $X\times Z$, and $X\times Y$, respectively, one has to say whether there is a measure $\mu$ over $X\times Y\times Z$ such that $\mu\pi_{Y\times Z}^{-1}=\mu_1$, $\mu\pi_{X\times Z}^{-1}=\mu_2$, and $\mu\pi_{X\times Y}^{-1}=\mu_3$, where $\pi_{Y\times Z}$ is the projection from $X\times Y\times Z$ to $Y\times Z$, etc.

This problem can be restated as follows. Let $U:=Y\times Z$. Then one has to say whether there is a measure $\mu$ over $X\times U$ such that $\mu\pi_X^{-1}=\mu_2\pi_{X\times Z\to X}^{-1}$ and $\mu\pi_{X\times U\to U}^{-1}=\mu_1$, with the additional affine restrictions specifying the $\mu$-distributions of the maps $X\times U\ni(x,u)\mapsto(x,\pi_{U\to Y}u)$ and $X\times U\ni(x,u)\mapsto(x,\pi_{U\to Z}u)$, where $\pi_{A\to B}$ is the projection from $A$ to $B$.