Skip to main content

New answers tagged

1 vote

trying to get intuition into why Cross Entropy will always be greater or equal to the Entropy

View the expression $\sum_{i\,=\,1}^n p_i \log(p_i/q_i)$ as a function of $(q_1,\ldots,q_n)$ with $(p_1,\ldots,p_n)$ fixed. You want to minimize this subject to the constraints $q_i\ge0$ for every $i$ ...
Michael Hardy's user avatar

Top 50 recent answers are included