0
$\begingroup$

I would like to know whether the following problem is well posed, and whether there is a solution. Let me make it clear that I have no pretentions of rigor here.

Given a continuum random variable $x$, I am looking for the probability distribution $p(x)$ which maximizes

\begin{equation} - \int_{-\infty}^{+ \infty} p(x) \log p(x) \, dx \end{equation} with the constraints \begin{eqnarray} \int_{-\infty}^{+\infty} p(x) \, \theta(x_{\ast}-x)\, dx &=& q,\\ \int_{-\infty}^{+\infty} p(x) \, dx &=& 1, \end{eqnarray} where $x_{\ast}$ and $0< q < 1$ are given numbers, and $\theta(x)$ is the Heaviside step function.

This is analogous to a maximum-entropy problem with a constraint on the cumulative distribution function (CDF) of $p(x)$, and the constraint that $p$ is normalized to unity.

I tried to solve the problem with Lagrange multipliers, but is not clear to me how to do it: I maximize

\begin{equation} \Lambda[p] \equiv - \int_{-\infty}^{+\infty} p(x) \log p(x) \, dx + \lambda \int_{-\infty}^{+\infty}p(x) \left[ \theta(x_{\ast}-x) - q \right]dx + \lambda' \left[ \int_{-\infty}^{+\infty} p(x) \, dx - 1 \right], \end{equation} where $\lambda$ and $\lambda'$ are Lagrange multipliers. I then take the derivative of $\Lambda$ with respect to $p(x)$, set it to zero, and obtain \begin{equation} p(x) = \frac{1}{Z} \exp\left\{ \lambda[\theta(x_{\ast}-x) - q \right]\} \end{equation} which is not normalizable.

Are you aware of where could be the issue? Or whether there is an alternative formulation of this problem which does not have this normalization issue?

Thank you

$\endgroup$
3
  • $\begingroup$ Normalization issue can be overcome if you restrict the support of $p(x)$ to $[0,\infty)$, and enforce $p(x)=0$ for $x<0$. This is the way an exponential distribution gets defined: en.wikipedia.org/wiki/… $\endgroup$ Feb 1, 2017 at 16:06
  • $\begingroup$ This does not solve the normalization issue: the result will still be $p(x) = 1/Z \exp\{ \lambda [\theta(x_{\ast} - x) - q]\}$, which does not converge for large $x$. Note that here $\theta(x)$ is the Heaviside step function. $\endgroup$
    – James
    Feb 1, 2017 at 21:24
  • $\begingroup$ Clearly if you allow the support to be $(-\infty, +\infty)$ then the integral of $p(x)$ is unbounded. So I think the only normalization option is to restrict the support to $[\alpha,\beta]$ such that $\alpha < x^{*} < \beta$, and then equate the sum of two resulting integrals to unity. $\endgroup$ Feb 1, 2017 at 22:18

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.