MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am currently going through the following article:

In this article, how did they arrive at the values in the Estimation step (Figure 1 Step b)? Specifically, how did they come up with values such as (0.45, 0.55), (0.80, 0.20), etc.? Any explanation is much appreciated.

share|cite|improve this question
Aryan, could I persuade you to change "EM" to "Expectation maximization" in your title? EM stands for various other things, and in any case it would help readers to spot quickly what field your question is in. Thanks. – Tom Leinster Dec 23 '09 at 13:25

These numbers are the normalized likelihoods that the results given in the 10 toss vector are obtained from the current distributions the coin A (or respectively B).

I'll work out the first two rows for illustration:

The guessed Bernoulli parameter for type A is 0.6 and for type B is 0.5. According to the binomial distribution formula, the unnormalized likelihood for obtaining 5H 5T are From A:

L_A = C(10,5)(0.6)^5(0.4)^5

where C(10,5) is the binomial coefficient 10!/5!5!

Similarly from B we obtain:

L_B = C(10,5)(0.5)^5(0.5)^5

The normalized likelihoods are obtained as

For A: L_A/(L_A+L_B) = 0.4491

For B: L_B/(L_A+L_B) = 0.5509

For the second case 9H 1T

L_A = C(10,9)(0.6)^9(0.4)^1

L_B = C(10,9)(0.5)^9(0.5)^9

The normalized likelihoods:

For A: L_A/(L_A+L_B) = 0.8050

For B: L_B/(L_A+L_B) = 0.1950

share|cite|improve this answer

The numbers in question are the conditional probabilities that each data set was generated using coin A or coin B given the number of heads that occured in the data set and the current estimate of each coin's bias. To compute these numbers, we calculate likelihoods of head counts given coin assignments and biases, then use Bayes' theorem to relate these likelihoods to the desired conditional probabilities.

As in the paper, let $\mathbf{z}$ be the current vector of (random variables representing) assignments of coin A or coin B to each data set, let $\mathbf{X}$ be the vector of head counts in each data set, and let $\theta = (\theta_A, \theta_B)$ be the current estimate of the coins' biases.

Then (for example) the first red number in Figure 1(b), step 2 is $\mathbb{P}(z_1 = A | X_1 = x_1, \theta).$

Apply Bayes' theroem to give $\mathbb{P}(z_1 = A | X_1 = x_1, \theta) = \frac{\mathbb{P}(X_1 = x_1 | z_1 = A, \theta) \mathbb{P}(z_1 = A | \theta)}{\mathbb{P}(X_1 = x_1 | \theta)}$.

Marginalize the denominator over $z_1$ and use the fact that $\mathbf{z}$ is assumed a priori to be uniformly distributed and independent of $\theta$ to cancel terms of the form $\mathbb{P}(z_1 | \theta)$ to give $\mathbb{P}(z_1 = A | X_1 = x_1, \theta) = \frac{\mathbb{P}(X_1 = x_1 | z_1 = A, \theta)}{\mathbb{P}(X_1 = x_1 | z_1 = A, \theta) + \mathbb{P}(X_1 = x_1 | z_1 = B, \theta)} = \frac{\theta_A^{x_1} (1 - \theta_A)^{n - x_1}}{\theta_A^{x_1} (1 - \theta_A)^{10 - x_1} + \theta_B^{x_1} (1 - \theta_B)^{10 - x_1}}$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.