# Estimating joint and conditional probabilities with incomplete information

I'm working on an application for which it would be great to have the following functionality:

Say that you have a collection $C$ of $n$ events, for now let's set $n = 3$ and call the events $a, b,$ and $c$

Given $k$, say that for any subset $K \subset C$ s.t. $||K|| = k < n$ of the set of events, you can produce the complete set of joint and conditional probabilities for any combination of the $k$ events. So, if $n = 3$ and $k = 2$, you would have complete information about

$p(a | b)$, $p(b | a)$, $p(a | \neg b)$, $p(a \wedge b)$, $p(\neg (a \wedge b))$, etc., etc.

$p(a | c)$, $p(c | a)$, $p(a | \neg c)$, $p(a \wedge c)$, $p(\neg (a \wedge c))$, etc., etc.

$p(c | b)$, $p(b | c)$, $p(c | \neg b)$, $p(c \wedge b)$, $p(\neg (c \wedge b))$, etc., etc.

Is there a 'correct' way to estimate the corresponding values for the joint and conditional probabilities for a, b, AND c?

Things such as $p(a \wedge b \wedge c)$, $p(\neg (a \wedge b \wedge c))$, $p(a | b \wedge c)$, $p(a \wedge b | c)$, etc., etc.

I am especially interested in the general case.

# Edit

I should have been more expansive in my questions, as regardless of whether or not there is a 'correct' way of answering this question, if there are methods that allow one to say anything interesting about the bounds, or anything interesting at all, really, about the joint & conditional probabilities of a, b, and c, than those methods will be very useful to me.

-

This paper addresses a similar problem I think, although I believe they consider binary outcomes only:

Ramsahai, R.R. (2007). Causal bounds and instruments. In Proceedings of the 23rd Annual Conference on Uncertainty in Artifical Intelligence, 310-317.

The main result is a way to produce the sorts of upper and lower probabilities that Douglas Zare mentions in his comment. They additionally note a freely available software package that they use, called polymake.

-

No, and to see that there is not, you might produce a collection of rademacher ($\pm 1$) random variables that are pairwise independent but not independent. No statement about 2 at a time, etc., would be different that for independent rademacher , but statements about the 3 way etc would be. You can do this, and generalise it to an N-K setting , by taking $X_1,X_2,X_3$ i.i.d. and $= \pm 1$ with prob $\frac 12$, and $Z_1 = X_2X_3, Z_2 = X_1X_3, Z_3 = X_1X_2$ .

-
You can still bound the probabilities above and below. –  Douglas Zare May 29 '12 at 21:25
Mike, thanks for the answer. That's what I anticipated. I don't follow the explanation 100%, but I understand the concept and probably just need more time to think about your example. Douglas, could you please elaborate a bit further on how you would bound the probabilities? –  tvladeck May 30 '12 at 20:48
There are multiple things you could mean by your question, and I thought you might want something more substantial, but since you accepted this answer I guess mike gave you what you want. –  Douglas Zare May 31 '12 at 19:15
Douglas, sorry that I have missed your comment until now. I am quite interested in a more substantial explanation of what can be said about the situation above, or anything at all - especially if the joint / conditional probabilities of a, b, and c can be bounded. I have edited the question to expand it. –  tvladeck Jun 12 '12 at 18:42