# Maximum entropy probability distribution with fixed interval and variance?

What is the maximum entropy probability distribution if the support is a fixed interval (e.g. $$[-1,1]$$) with an already known variance?

If we know the support is a fixed interval, then the maximum entropy prob. distribution is the uniform distribution. But if we also add the variance constraint, I'm not sure what the answer is.

Thanks!

• If you want to fix the (bounded) interval and mean, not interval and variance, the answer is given at mathoverflow.net/questions/116667/… Commented Apr 14 at 15:42

Denote the support by $$[a,b]$$ and the variance by $$\sigma^2$$. Repeating the variational calculus argument that shows the normal distribution maximizes the differential entropy (see here for instance), probability distributions of variance $$\sigma^2$$ and supported in $$[a,b]$$ that attain the largest possible entropy should have a density function of the form

$$p(x)=\frac{e^{-\lambda(x-\mu)^2}}{\int_a^be^{-\lambda(t-\mu)^2}{\rm{d}}t}\quad (a\leq x\leq b)\tag{\star}$$ where $$\lambda$$ and $$\mu$$ satisfy

$$\begin{cases} \int_a^bte^{-\lambda(t-\mu)^2}{\rm{d}}t=\mu\int_a^be^{-\lambda(t-\mu)^2}{\rm{d}}t,\\ \int_a^b(t-\mu)^2e^{-\lambda(t-\mu)^2}{\rm{d}}t=\sigma^2\int_a^be^{-\lambda(t-\mu)^2}{\rm{d}}t. \end{cases}$$ The first equation holds automatically if $$\mu$$ is at the center of the interval $$(a,b)$$. Assuming that $$a,b$$ are finite and the variational problem has a unique solution, then $$\mu$$ must be $$\frac{a+b}{2}$$: If $$p:[a,b]\rightarrow(0,\infty)$$ is the density function for the maximizer, then $$x\in [a,b]\mapsto p(a+b-x)$$ defines another probability distribution with the same support, variance and differential entropy. Then, the first equation above becomes irrelevant and one only needs to solve $$\int_a^b\left(t-\frac{a+b}{2}\right)^2e^{-\lambda\left(t-\frac{a+b}{2}\right)^2}{\rm{d}}t=\sigma^2\int_a^be^{-\lambda\left(t-\frac{a+b}{2}\right)^2}{\rm{d}}t \tag{\star\star}$$ for $$\lambda$$.

Update) Following the comment by KConrad, a more general system of equations should be considered. Let's write the density function as

$$p(x)=\frac{e^{-(\lambda x^2+\beta x)}}{\int_a^be^{-(\lambda x^2+\beta x)}{\rm{d}}t}\quad (a\leq x\leq b), \tag{\star\star\star}$$ In $$(\star)$$, it was implicitly assumed that $$-\frac{\beta}{2\lambda}$$ coincides with the mean $$\mu$$. Given the form $$(\star\star\star)$$ for the density function, the system of equations becomes

$$\begin{cases} \int_a^bte^{-(\lambda t^2+\beta t)}{\rm{d}}t= \mu\int_a^be^{-(\lambda t^2+\beta t)}{\rm{d}}t,\\ \int_a^b(t-\mu)^2e^{-(\lambda t^2+\beta t)}{\rm{d}}t= \sigma^2\int_a^be^{-(\lambda t^2+\beta t)}{\rm{d}}t. \end{cases}$$

As argued before, assuming that the solution is unique, $$p(x)$$ must coincide with $$p(a+b-x)$$. When $$p(x)$$ is of the form $$(\star\star\star)$$, by comparing the coefficients of $$x$$ in the exponent from the numerator, $$p(a+b-x)\equiv p(x)$$ implies $$-\frac{\beta}{2\lambda}=\frac{a+b}{2}$$. Moreover, $$\int_a^btp(t){\rm{d}}t=\int_a^btp(a+b-t){\rm{d}}t$$ implies that the mean $$\mu=\int_a^btp(t){\rm{d}}t$$ is equal to $$\frac{a+b}{2}$$. Just like before, when $$\mu=-\frac{\beta}{2\lambda}=\frac{a+b}{2}$$, the first equation holds automatically because $$\int_a^b\left(t-\frac{a+b}{2}\right)e^{-\lambda\left(t-\frac{a+b}{2}\right)^2}{\rm{d}}t=0$$. So it only remains the second equation which would be same as $$(\star\star)$$, and should be solved for $$\lambda$$ to derive a density function of the form $$p(x)=\frac{e^{-\lambda\left(x-\frac{a+b}{2}\right)^2}}{\int_a^be^{-\lambda\left(t-\frac{a+b}{2}\right)^2}{\rm{d}}t}.$$

• In your equation to determine the mean, you are using $\mu$ in two different ways: the $\mu$ in the exponent need not be the mean $\mu$ outside the integral on the right side. On the interval $[0,1]$ with $p(x) = e^{-(x-1/3)^2}/\int_0^1 e^{-(x-1/3)^2}\,dx$, the mean $\int_0^1 xp(x)\,dx$ is around $.47407$, not $1/3$. Commented Apr 14 at 14:39
• So just the normal distribution conditioned on being within this interval? Commented Apr 14 at 15:49
• This may not work if the known variance $\sigma^2 > \frac{(b-a)^2}{12}$ unless you allow $\lambda$ to go negative; if you do, this would not be a truncated normal distribution Commented Apr 27 at 9:59