Consider an experiment consisting of a repeated trial with two random Bernoulli (=binary) variables, A and B. Each trial consists of multiple outcomes for both A and B. Each trial has the same number of samples and the underlying joint distribution of A and B is the same everywhere.

Of course it's not possible to infer the joint probability distribution for a single trial, since we don't assume A and B to be independent.

But is it possible to make an estimation of the joint probability distribution from the set of marginals we obtained?

Just to make things clearer: here's an example.

  • Trial 1 gives: p(a=0) = .3 (so p(a=1)=.7) ; p(b=0) = .2 (so p(b=1)=.8)
  • Trial 2 gives: p(a=0) = .1 ; p(b=0) = .5
  • Trial 3 gives: p(a=0) = .7 ; p(b=0) = .9
  • Trial 4 gives: p(a=0) = .4 ; p(b=0) = .6

My question is: how can I estimate p(a=0,b=0), p(a=0,b=1), p(a=1,b=0) and p(a=1,b=1) ?

Remember that I don't have access to the individual results of each sample in a trial - I only know the marginals of each trial.

  • $\begingroup$ You can get a simple estimator from the average of $p(a=1)$, the average of $p(b=1)$, and the correlation between $\vec{p(a=1)}$ and $\vec{p(b=1)}$. However, it's not clear that this estimate is consistent as the number of samples goes to infinity unless you also increase the sample size. $\endgroup$ – Douglas Zare May 20 '13 at 21:26
  • $\begingroup$ As the number of samples goes to infinity, the variance in the marginals will go to 0, as will their covariance. The correlation will then converge to 0/0, which is undefined. The sample size needs to be in some relation to the number of samples; it will be some monotone increasing function which I will not figure out. I tried your method and tests with a sample size of 100 and with 5000 samples give an average absolute error of .0033, which is not that good. Your method assumes the correlation corr(p(a=1), p(b=1)) is a good estimator of corr(a,b), but is this justified? I need proofs! $\endgroup$ – Angelorf Jun 19 '13 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.