It seems that the derivations of the maximum entropy distributions is a "well-known" fact and so it is in the continuous and discrete cases.... However, I can't seem to find a proof/formal statement in the general case on a general probability space $(\Omega,\mathcal{F},\mathbb{P})$?

The proof in the continuous case can be found in Theorem 12.1.1 here.. but what about the general case for an arbitrary Borel reference measure?

  • $\begingroup$ In my version of the book theorem 12.1.1 is about the method of types rather than maximum entropy, whereas chapter 11 is all about the principle of maximum entropy principle - is that what you meant? $\endgroup$ – Paul Siegel Aug 9 at 17:31
  • $\begingroup$ (I was chasing your reference because I couldn't tell if you were asking about the derivation of the principle of maximum entropy itself or of specific distributions that satisfy maximum entropy principles according to given constraints. I might be able to come up with examples of the latter, but I think the former would be difficult except in a relative sense.) $\endgroup$ – Paul Siegel Aug 9 at 17:34
  • $\begingroup$ There should be no substantial difference in the derivation between (say) the "continuous" case, of the Lebesgue reference measure, and the general case . Essentially the same derivation as in the "continuous" case, almost verbatim, based on Lagrange multipliers/dual problem, should work for any reference measure. $\endgroup$ – Iosif Pinelis Aug 9 at 17:39
  • $\begingroup$ Yes I ended up proving it in the general care; verbatim... However, it's not completely clear to me the conditions in which a maximum entropy distribution (with prespeified moment constraints) should excist $\endgroup$ – AIM_BLB Aug 9 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.