# Can you explain a step in an expectation maximization algorithm in a Nature article?

I am currently going through the following article: http://www.nature.com/nbt/journal/v26/n8/full/nbt1406.html

In this article, how did they arrive at the values in the Estimation step (Figure 1 Step b)? Specifically, how did they come up with values such as (0.45, 0.55), (0.80, 0.20), etc.? Any explanation is much appreciated.

-
Aryan, could I persuade you to change "EM" to "Expectation maximization" in your title? EM stands for various other things, and in any case it would help readers to spot quickly what field your question is in. Thanks. –  Tom Leinster Dec 23 '09 at 13:25

These numbers are the normalized likelihoods that the results given in the 10 toss vector are obtained from the current distributions the coin A (or respectively B).

I'll work out the first two rows for illustration:

The guessed Bernoulli parameter for type A is 0.6 and for type B is 0.5. According to the binomial distribution formula, the unnormalized likelihood for obtaining 5H 5T are From A:

L_A = C(10,5)(0.6)^5(0.4)^5

where C(10,5) is the binomial coefficient 10!/5!5!

Similarly from B we obtain:

L_B = C(10,5)(0.5)^5(0.5)^5

The normalized likelihoods are obtained as

For A: L_A/(L_A+L_B) = 0.4491

For B: L_B/(L_A+L_B) = 0.5509

For the second case 9H 1T

L_A = C(10,9)(0.6)^9(0.4)^1

L_B = C(10,9)(0.5)^9(0.5)^9

The normalized likelihoods:

For A: L_A/(L_A+L_B) = 0.8050

For B: L_B/(L_A+L_B) = 0.1950

-

The numbers in question are the conditional probabilities that each data set was generated using coin A or coin B given the number of heads that occured in the data set and the current estimate of each coin's bias. To compute these numbers, we calculate likelihoods of head counts given coin assignments and biases, then use Bayes' theorem to relate these likelihoods to the desired conditional probabilities.

As in the paper, let $\mathbf{z}$ be the current vector of (random variables representing) assignments of coin A or coin B to each data set, let $\mathbf{X}$ be the vector of head counts in each data set, and let $\theta = (\theta_A, \theta_B)$ be the current estimate of the coins' biases.

Then (for example) the first red number in Figure 1(b), step 2 is $\mathbb{P}(z_1 = A | X_1 = x_1, \theta).$

Apply Bayes' theroem to give $\mathbb{P}(z_1 = A | X_1 = x_1, \theta) = \frac{\mathbb{P}(X_1 = x_1 | z_1 = A, \theta) \mathbb{P}(z_1 = A | \theta)}{\mathbb{P}(X_1 = x_1 | \theta)}$.

Marginalize the denominator over $z_1$ and use the fact that $\mathbf{z}$ is assumed a priori to be uniformly distributed and independent of $\theta$ to cancel terms of the form $\mathbb{P}(z_1 | \theta)$ to give $\mathbb{P}(z_1 = A | X_1 = x_1, \theta) = \frac{\mathbb{P}(X_1 = x_1 | z_1 = A, \theta)}{\mathbb{P}(X_1 = x_1 | z_1 = A, \theta) + \mathbb{P}(X_1 = x_1 | z_1 = B, \theta)} = \frac{\theta_A^{x_1} (1 - \theta_A)^{n - x_1}}{\theta_A^{x_1} (1 - \theta_A)^{10 - x_1} + \theta_B^{x_1} (1 - \theta_B)^{10 - x_1}}$.

-