The von Mises distribution has the highest entropy for a given first circular moment. It seems that the converse is true: for a fixed entropy, the von Mises distribution has the highest first circular moment (in magnitude). This seems like something that should already be proven somewhere, but I haven't been able to find it, nor have my efforts at proving it met with success so far. Does anyone know where a proof of this is (or failing that, how to prove it)?
-
$\begingroup$ It would algebraically make sense. Maximising the magnitude of the first circular moment with fixed $H=-\int_\Gamma P\log P\,d\theta$ and $1=\int_\Gamma P\,d\theta$ would yield something like $f(\cos\theta)-\lambda_1(1+\log P)-\lambda_2=0$ for some $f$ and the $e^{f(\cos\theta)}$ term should hopefully match the von Mises pdf. $\endgroup$– TheSimpliFireCommented Jun 18, 2023 at 9:00
-
$\begingroup$ Sorry for the delay; I was busy loading all my stuff onto a truck and sending it across the country yesterday. Your answer is beautiful! I was trying to establish the claim starting with the distribution being optimal for the mean and proving by contradiction, but it was getting me into inequalities of ratios of Bessel functions, and not going anywhere. The direct approach you took is vastly cleaner. Thank you! $\endgroup$– Jonathan ShawCommented Jun 21, 2023 at 3:54
1 Answer
Yes, this is true.
Indeed, the first circular moment for a probability density $f$ on the interval $[0,2\pi]$ is $$m_1(f):=\int_0^{2\pi}e^{ix}f(x)\,dx.$$ So, $$\begin{aligned}&|m_1(f)| \\ &=\max_{u\in[0,2\pi]}\Big(\cos u\,\int_0^{2\pi}\cos x\,f(x)\,dx +\sin u\,\int_0^{2\pi}\sin x\,f(x)\,dx\Big) \\ &=\max_{u\in[0,2\pi]}\int_0^{2\pi}\cos(x-u)\,f(x)\,dx. \end{aligned}$$
So, a density $f$ maximizing $|m_1(f)|$ given the entropy $H:=-\int_0^{2\pi}f\ln f$ is maximizing $$I_u(f):=\int_0^{2\pi}\cos(x-u)\,f(x)\,dx$$ for some $u\in[0,2\pi]$, again given the entropy. Using Lagrange multipliers, we see that for such a maximizer $f$ we have $$\rho\cos(x-u)=\lambda\ln f(x)+\mu$$ for some real $\rho,\lambda,\mu$ with $\rho^2+\lambda^2+\mu^2>0$ and almost all (a.a.) $x\in[0,2\pi]$. So, for some real $a$ and $b$ and a.a. $x\in[0,2\pi]$ we have $\ln f(x)=a\cos(x-u)+b$ and hence $f(x)=ce^{a\cos(x-u)}$ for $c:=e^b$. Thus, $f$ is a von Mises density. $\quad\Box$