3
$\begingroup$

Given constants $m_0,\ldots m_n$ and a measure $\mu$ on $\mathbb{R}$, how can I "recover" the integrals $\int f x^n d\mu$ of the maximum entropy distribution $f\in L^2(\mathbb{R})$ which satisfies $\int f^2 x^n d\mu=m_n$ ?

$\endgroup$
3
$\begingroup$

Let us consider the special case when $\mu$ is the Lebesgue measure on $\mathbb{R}$. The derivation then should be extendable to the general case without much effort.

We need to find the distribution $f$ that solves the following problem: $$\max_{f\in \mathcal{B}}-\int_\mathbb{R}f(x)\ln f(x) dx\\\Leftrightarrow\min_{f\in \mathcal{B}}\int_\mathbb{R}f(x)\ln f(x) dx\\\mathrm{s.t.}\quad \int_\mathbb{R}f(x) dx=1,\ \int_\mathbb{R}f^2(x) x^kdx=m_k,\ k=0,1,\cdots,\ n$$ where $\mathcal{B}$ is the set of Borel measurable functions on $\mathbb{R}$. The corresponding Lagrangian is $$\mathcal{L}\left(f(x),\{\lambda_k\}_{k=0}^n\right)=\int_\mathbb{R}f(x)\ln f(x) dx-\lambda_0\int_\mathbb{R}f(x) dx-\sum_{k=1}^n \lambda_k \int_\mathbb{R}f^2(x) x^kdx$$ The maximizing distribution is obtained by equating the derivative of the Lagrangian to $0$ $$\frac{\partial \mathcal{L}}{\partial f}=1+\ln f(x)-\lambda_0-2\sum_{k=1}^n\lambda_k f(x) x^k=0\\\implies \ln f(x)=\lambda_0-1+2f(x)\sum_{k=1}^n \lambda_k x^k\\\implies f(x)=a\exp\left(bf(x)\right)\\\implies f(x)=-\frac{W(-ab)}{b}$$ where $W$ is the Lambert's W function, $a=e^{\lambda_0-1},\ b=2\sum_{k=1}^n \lambda_k x^k$ Thus, we can express the integral $\int_\mathbb{R} f(x) x^k dx$ as below: $$\int_\mathbb{R} f(x) x^k dx=-\int_\mathbb{R}\frac{W\left(-2a\sum_{j=1}^n \lambda_j x^j\right)}{2\sum_{j=1}^n \lambda_j x^j}x^k dx$$ Beyond this point, the integral seems to be formidable, and probably a numerical approach is required, but in principle, this derivation produces some form of the maximizing distribution $f$, which one can then work with.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.