Conditional probability and independence - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-24T10:07:33Z http://mathoverflow.net/feeds/question/91608 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/91608/conditional-probability-and-independence Conditional probability and independence unknown (google) 2012-03-19T11:08:01Z 2012-03-26T17:48:31Z <p>Suppose that we have vectors of events <code>$\{H_1,...,H_n\}$</code> and <code>$\{D_1,...,D_m\}$</code>. Consider the following two sets of conditions:</p> <p><strong>Condition set 1</strong></p> <p>(1) $P(H_i H_j)=0$ for any $i\neq j$ and $\sum_iP(H_i)=1$ </p> <p>(2) $P(D_1D_2...D_m|H_i)=\prod_jP(D_j|H_i)$, $1\leq i\leq n$</p> <p>(3) $P(D_1D_2...D_m|\overline{H_i})=\prod_jP(D_j|\overline{H_i})$, $1\leq i\leq n$</p> <p>where $\overline{H_i}$ means the negation of $H_i$.</p> <p><strong>Condition set 2</strong></p> <p>(1) $P(H_i H_j)=0$ for any $i\neq j$ and $\sum_iP(H_i)=1$ </p> <p>(2) $P(D_rD_s|H_i)=P(D_r|H_i)P(D_s|H_i)$, for any $r\neq s$, $1\leq i\leq n$</p> <p>(3) $P(D_rD_s|\overline{H_i})=P(D_r|\bar{H_i})P(D_s|\bar{H_i})$, for any $r\neq s$, $1\leq i\leq n$</p> <p><strong>Claim:</strong> If $n>2$, then at most one of the following fractions </p> <p>$\frac{P(D_1|H_i)}{P(D_1|\overline{H_i})},\frac{P(D_2|H_i)}{P(D_2|\overline{H_i})},...,\frac{P(D_m|H_i)}{P(D_m|\overline{H_i})}$ can differ from unity, $1\leq i\leq n$.</p> <p><strong>The question:</strong> Is the 1st set of conditions sufficient to establishing the claim, or only the 2nd set of stronger conditions is sufficient? Either way, how can we prove it? Is there an intuitive explanation why the claim has to be true giving those conditions? </p> <hr> <p>To understand the background and motivation for this problem, think of ${H_1,...,H_n}$ as a set of exhuastive and mutually independent (which means (1) applies) candidate hypotheses we want to test by some experiment that generates data ${D_1,...,D_m}$. </p> <p>Define $O(H_i|D_1D_2...D_m)\equiv \frac{P(H_i|D_1D_2...D_m)}{P(\bar{H_i}|D_1D_2...D_m)}$ as the <strong><em>Odds</em></strong> that $H_i$ is true v.s false, giving data $D_1$ to $D_m$. </p> <p>By this definition, $O(H_i|D_1D_2...D_m)=O(H_i)\frac{P(D_1D_2...D_m|H_i)}{P(D_1D_2...D_m|\bar{H_i})}$.</p> <p>Now in such tests it is common that you can design your experiment so that the data $D_i$ are mutually independent, giving $H_i$. So (2) is true.</p> <p>If the claim is true and you have more than two hypotheses to test, the experiment will serve its purpose only if (3) is false. To see why, suppose instead (3) holds, so the data are also independent giving the negation of $H_i$. Then we have</p> <p>$O(H_i|D_1D_2...D_m)=O(H_i) \prod_j\frac{P(D_j|H_i)}{P(D_j|\bar{H_i})}$ (*)</p> <p>But by the claim, only at most one of the fractions in the production can differ from $1$, which means at most one datum can be useful for improving upon the prior odds of a hypothesis ($O(H_i)$). </p> <p>The lesson is that giving (1) and (2), even if $D_i$'s are physically or causally independent, (3) remains a strong ad hoc assumption that either reduces the information of additional data to triviality (if true) or calculates incorrect result by (*) (if false). </p> <hr> <p>Source: E.T.Jaynes, Probability theory, exercise 4.1. </p> http://mathoverflow.net/questions/91608/conditional-probability-and-independence/92296#92296 Answer by Arthur B for Conditional probability and independence Arthur B 2012-03-26T17:48:31Z 2012-03-26T17:48:31Z <p>A nice proof is given here, <a href="http://research.microsoft.com/pubs/66913/multiple-hypothesis.ps" rel="nofollow">http://research.microsoft.com/pubs/66913/multiple-hypothesis.ps</a></p>