Timeline for Numeric problem when evaluating log of a pdf
Current License: CC BY-SA 2.5
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 31, 2010 at 19:44 | comment | added | Dylan Thurston | To approximate the log of a sum, you can use a power-series approximation, or perhaps more easily, just pull out a constant term. See here: lingpipe-blog.com/2009/06/25/log-sum-of-exponentials for more description. | |
| Oct 31, 2010 at 17:40 | comment | added | eakbas | What if $p(x)$ is a mixture model, e.g. a Gaussian mixture: $p(x;\theta) = \sum_{i=1}^N \pi_i f(x;\mu_i, \Sigma_i)$? You cannot express $log(p(x))$ analytically, and yet you might have some of the $p(\cdot)$ values very close to zero. | |
| Oct 27, 2010 at 2:48 | vote | accept | eakbas | ||
| Oct 27, 2010 at 2:47 | comment | added | eakbas | My bad, I was NOT directly using logs. Thanks, Dylan. | |
| Oct 24, 2010 at 5:46 | comment | added | Dylan Thurston | Can you say more about the setting? Your initial wording suggests that you compute p(x), and then take the log, which is the wrong thing to do. Rather, just find log(p(x)) directly. Eg, if x is a sequence of events and p(x) is the product of their individual probabilities, compute the logs of each of those individual probabilities (which should safely avoid underflow), and then add the logs. But maybe you're doing something else? | |
| Oct 24, 2010 at 5:33 | comment | added | eakbas | I am already working directly with logs of probability. But still I have this problem. Maybe I should consider those points which cause this problem as outliers and ignore them. Or somebody suggested that I add a very small constant to the probability values (this is possible if you are computing p(x)'s first.) | |
| Oct 23, 2010 at 14:28 | history | edited | Dylan Thurston | CC BY-SA 2.5 |
edited body
|
| Oct 23, 2010 at 14:24 | comment | added | J. M. isn't a mathematician | I was curious because my experience with nonconverging iterations as presented to me mostly turned out to be the user providing piss-poor initial estimates. That being said, I see your point. Thanks! | |
| Oct 23, 2010 at 13:59 | history | answered | Dylan Thurston | CC BY-SA 2.5 |