I am not sure if the question is appropriate but I want to try my luck. One can estimate a parameter using maximum likelihood and we know it is optimal. On the other hand there are methods which uses maximum entropy to estimate parameter. Can someone compare these two methods? My intuition is that there are cases where both are equivalent. Is there a way to find when these two methods are equivalent and when not?
$\begingroup$
$\endgroup$
4
-
$\begingroup$ I am not sure what the question is really asking, but perhaps you are after the "duality" relation; see e.g., theorem 7 in alex.smola.org/papers/2006/AltSmo06.pdf $\endgroup$– SuvritDec 19, 2015 at 22:59
-
$\begingroup$ @Suvrit Thank you for your comments and link. I will read the link. It seems both are examined through exponential family but these methods are valid for any distribution. My question is relate to the link arxiv.org/pdf/math/0009129.pdf $\endgroup$– CreatorDec 20, 2015 at 0:10
-
$\begingroup$ @Creator : for an approach via maximum likelihood you can check the book of Iacus there is a R implementation, although it does'nt tackle the maximum entropy approach (I don't know if such an approach is even fully developed) . Best regards $\endgroup$– The BridgeDec 20, 2015 at 22:10
-
$\begingroup$ @TheBridge, Thank you for your interest. I am more interested to know when they lead to different answers. $\endgroup$– CreatorDec 21, 2015 at 0:21
Add a comment
|