0
$\begingroup$

I am not sure if the question is appropriate but I want to try my luck. One can estimate a parameter using maximum likelihood and we know it is optimal. On the other hand there are methods which uses maximum entropy to estimate parameter. Can someone compare these two methods? My intuition is that there are cases where both are equivalent. Is there a way to find when these two methods are equivalent and when not?

$\endgroup$
4
  • $\begingroup$ I am not sure what the question is really asking, but perhaps you are after the "duality" relation; see e.g., theorem 7 in alex.smola.org/papers/2006/AltSmo06.pdf $\endgroup$
    – Suvrit
    Dec 19, 2015 at 22:59
  • $\begingroup$ @Suvrit Thank you for your comments and link. I will read the link. It seems both are examined through exponential family but these methods are valid for any distribution. My question is relate to the link arxiv.org/pdf/math/0009129.pdf $\endgroup$
    – Creator
    Dec 20, 2015 at 0:10
  • $\begingroup$ @Creator : for an approach via maximum likelihood you can check the book of Iacus there is a R implementation, although it does'nt tackle the maximum entropy approach (I don't know if such an approach is even fully developed) . Best regards $\endgroup$
    – The Bridge
    Dec 20, 2015 at 22:10
  • $\begingroup$ @TheBridge, Thank you for your interest. I am more interested to know when they lead to different answers. $\endgroup$
    – Creator
    Dec 21, 2015 at 0:21

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.