Estimating joint and conditional probabilities with incomplete information - MathOverflow

Estimating joint and conditional probabilities with incomplete information

2012-05-29T19:04:18Z

I'm working on an application for which it would be great to have the following functionality:

Say that you have a collection $C$ of $n$ events, for now let's set $n = 3$ and call the events $a, b,$ and $c$

Given $k$, say that for any subset $K \subset C$ s.t. $||K|| = k < n$ of the set of events, you can produce the complete set of joint and conditional probabilities for any combination of the $k$ events. So, if $n = 3$ and $k = 2$, you would have complete information about

$p(a | b)$, $p(b | a)$, $p(a | \neg b)$, $p(a \wedge b)$, $p(\neg (a \wedge b))$, etc., etc.

$p(a | c)$, $p(c | a)$, $p(a | \neg c)$, $p(a \wedge c)$, $p(\neg (a \wedge c))$, etc., etc.

$p(c | b)$, $p(b | c)$, $p(c | \neg b)$, $p(c \wedge b)$, $p(\neg (c \wedge b))$, etc., etc.

Is there a 'correct' way to estimate the corresponding values for the joint and conditional probabilities for a, b, AND c?

Things such as $p(a \wedge b \wedge c)$, $p(\neg (a \wedge b \wedge c))$, $p(a | b \wedge c)$, $p(a \wedge b | c)$, etc., etc.

I am especially interested in the general case.

Edit

I should have been more expansive in my questions, as regardless of whether or not there is a 'correct' way of answering this question, if there are methods that allow one to say anything interesting about the bounds, or anything interesting at all, really, about the joint & conditional probabilities of a, b, and c, than those methods will be very useful to me.

Answer by mike for Estimating joint and conditional probabilities with incomplete information

2012-05-29T21:20:53Z

No, and to see that there is not, you might produce a collection of rademacher ($\pm 1$) random variables that are pairwise independent but not independent. No statement about 2 at a time, etc., would be different that for independent rademacher , but statements about the 3 way etc would be. You can do this, and generalise it to an N-K setting , by taking $X_1,X_2,X_3$ i.i.d. and $= \pm 1$ with prob $\frac 12$, and $Z_1 = X_2X_3, Z_2 = X_1X_3, Z_3 = X_1X_2$ .

Answer by R Hahn for Estimating joint and conditional probabilities with incomplete information

2012-06-12T19:02:06Z

This paper addresses a similar problem I think, although I believe they consider binary outcomes only:

Ramsahai, R.R. (2007). Causal bounds and instruments. In Proceedings of the 23rd Annual Conference on Uncertainty in Artifical Intelligence, 310-317.

The main result is a way to produce the sorts of upper and lower probabilities that Douglas Zare mentions in his comment. They additionally note a freely available software package that they use, called polymake.