Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

$X\in \mathbb{R}$. Which distribution $P(X)$ has the highest possible entropy given its expected value, variance, skewness, and kurtosis? Is it an exponential family distribution of the form $P(X) \propto \exp(a\cdot x +b\cdot x^2 + c\cdot x^3 +d\cdot x^4)$ in analogy to the normal distribution being the maximum-entropy distribution given mean and variance? If so, where can I read about it?

I am interested in this question because I want to model data with high skewness and high kurtosis, but I am still looking for an appropriate distribution.

share|improve this question
    
Perhaps better suited for stats.SE –  Memming Sep 10 '13 at 4:17
7  
Probability theory is certainly on-topic here. –  Robert Israel Sep 10 '13 at 5:06

1 Answer 1

up vote 5 down vote accepted

Yes, your $P(X) \propto \exp(a\cdot x +b\cdot x^2 + c\cdot x^3 +d\cdot x^4)$ maximises the entropy $-\int P(X){\rm log} P(X)dX$ for prescribed first four moments, if the skewness and curtosis lie in a certain range:

M. Rockinger and E. Jondau, Entropy densities with an application to autoregressive conditional skewness and kurtosis (2002).

This simple generalization of the normal distribution holds also if higher moments are prescribed, provided that the highest prescribed moment is even. If, for example, only mean, variance, and skewness, are prescribed, then $d$ would be zero and the distribution fails to normalize. In this case the maximum entropy distribution exists, but it has a more complicated form, as discussed here.

You'll still have to determine the coefficients $a,b,c,d$ from the given first four moments. The cited 2002 paper gives a description of an efficient method. There may be no solution (typically, if the skewness is too large relative to the kurtosis), meaning that a maximum entropy solution of this simple form does not exist (see Figure 1 of the paper). The combination of a prescribed skewness and kurtosis typically leads to a bimodal distribution (see Figures 2 and 3), which may or may not be desirable for your application.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.