Hello,

I’m having some problems in ensuring the non-negativity of **KLD**!

I know that **KLD** is always positive and I went over the proof. However, it doesn’t seem to work for me. In some cases I’m getting negative results. Here is how I’m using **KLD**:

KLD( P(x) || Q(x) ) = Sum[ (P(x) . Log[P(x)/Q(x)] ]; Log is base 2. P(x) and Q(x) are two different distributions for all x belongs to X.

For example, P(x) = (Frequency(x)/Total_Size); just normal PMF! The same thing for Q(x). Note that the Total_Size for P might be different from Q distribution.

Could you please let me know if I’m missing something? Are there any special conditions that I have to take into consideration to avoid having negative results?

Thanks in advance.