# How to use Dirichlet Prior

Regarding this question: How to compute KL-divergence when PMF contains 0s?

One of the solutions requires the use of a "Dirichlet Prior", I am not exactly sure how this works.

1) Does the dirichlet prior change the result of the kl-divergence as opposed to not using it? (assuming that the input is numerically stable)

2) Should the dirichlet prior be applied on just Q or both P and Q? The kl-divergence is defined for all P_i = 0

3) Exactly how is the dirichlet prior applied? For example, if I have the data-set

P: {0.4, 0.6} Q: {1.0, 0.0}

Then would this be correct?

dirichlet(P) = { (0.4 + 1)/( (0.4 + 0.6) + 2 ) , (0.6 + 1)/( (0.4+0.6) + 2 ) } = { 0.466... , 0.5333... }

dirichlet(Q) = { (1 + 1)/( (0 + 1) + 2 ) , (0 + 1)/( (0 + 1) + 2 ) } = { 0.666, 0.333 }

edit: Regarding #2 looking at numerical results I -think- the prior must be applied on both P and Q. Using the following:

P: {0.4, 0.6} Q: {1.0, 2.0}

dirichlet(P) = { (0.4 + 1)/( (0.4 + 0.6) + 2 ) , (0.6 + 1)/( (0.4+0.6) + 2 ) } = { 0.466... , 0.5333... }

dirichlet(Q) = { (1 + 1)/( (2 + 1) + 2 ) , (2 + 1)/( (2 + 1) + 2 ) } = { 0.4, 0.6 }

kl(P, Q) = 0.00971231

kl(d(P), d(Q)) = 0.00911929

kl(P, d(Q)) = 0 // pretty far from the expected result.

-