2
$\begingroup$

I'm hoping for some help in nailing down a vague idea about independence. It starts with finding the expectation of a product of three RVs (or more, but I'll stick to three for now). These are not necessarily independent, but we know all the pairwise covariances. This is enough information to predict the expected value of the product of any pair, but it is not enough to predict the expected value of the product of all three, as can be shown by an example. Let A, B, and C be three Bernoulli RVs: each takes on the values 0 and 1 with equal probability. Further, assume that they are independent. Obviously $\mathbb{E}\left[ABC\right]=1/8$. Define a new variable $Z=A(1-B)+B(1-A)$, that is, $Z=1$ iff $A\ne B$. Z is independent of A, and it is independent of B. In fact, all the pairwise joint distributions of ${A,B,C}$ are identical to the pairwise joint distributions of ${A,B,Z}$. The joint distributions of the three taken together are different, of course. And, in particular, $A=B=Z=1$ never happens, and $\mathbb{E}\left[ABZ\right]=0\ne \mathbb{E}\left[ABC\right]$. This shows that knowledge of all the pairwise joint distributions of three or more RVs is not sufficient to predict the expected value of their product.

Is there a way, for Bernoulli RVs or more generally, to define the distribution that has specified single-variable (p's for each variable) and pairwise (correlations for each pair) statistics, but has no higher-level correlations? I know that I'm being vague--what do I mean by "higher-level correlations"? But that's the problem: I can't think how to define it. The idea is something like conditional independence, but I haven't been able to make it precise.

$\endgroup$
1

1 Answer 1

4
$\begingroup$

Higher order correlations are called joint cumulants or semi-invariants. They can be expressed in terms of joint moments. If you want one of the cumulants to be equal to zero, that gives you an algebraic equation on moments. A joint moment of Bernoulli r.v.'s equals the probability that the r.v.'s involved are all equal to $1$. Combining all that one obtains a system of algebraic equations on the set of probability distributions on the binary cube. That needs to be solved. So, the more cumulants you want to be zero, the more equations you have, so the system may easily be overdefined.

$\endgroup$
3
  • 1
    $\begingroup$ I think your first sentence may be the answer. If I understand you correctly, for three Bernoulli RVs with mean p and covariance c, the joint distribution with no higher-level correlations would have k(i,0,0)=k(0,i,0)=k(0,0,i)= the Bernoulli cumulants, k(1,1,0)=k(0,1,0)=k(0,0,1)=c, and k(i,j,k)=0 for any i+j+k>2 with more than 1 non-zero. So now I can write the cumulant generating function and I have defined the distribution. But I'm finished now, right? I understand that there is a relationship, but why do I need to express the cumulants in terms of moments? $\endgroup$
    – Leon Avery
    Jul 4, 2014 at 19:39
  • 1
    $\begingroup$ I made a guess that moments were a more familiar notion for you, and used them to argue that the equations you will get are always polynomial, but yes, you are right that you may do all this without mentioning moments at all. $\endgroup$ Jul 4, 2014 at 21:34
  • $\begingroup$ For my immediate problem, I need to be able to calculate the expectation of a product of three (or maybe more) correlated Bernoulli RVs, where I know all the pairwise stats. I can calculate that just knowing the cumulants (thanks, Wikipedia!). So if I can just assume all the higher three-way cumulants are zero, I think I have what i need. Spasebo! $\endgroup$
    – Leon Avery
    Jul 4, 2014 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.