0
$\begingroup$

I would like to know whether the following problem is well posed, and whether there is a solution. Let me make it clear that I have no pretentions of rigor here.

Given a continuum random variable $x$, I am looking for the probability distribution $p(x)$ which maximizes

\begin{equation} - \int_{-\infty}^{+ \infty} p(x) \log p(x) \, dx \end{equation} with the constraints \begin{eqnarray} \int_{-\infty}^{+\infty} p(x) \, \theta(x_{\ast}-x)\, dx &=& q,\\ \int_{-\infty}^{+\infty} p(x) \, dx &=& 1, \end{eqnarray} where $x_{\ast}$ and $0< q < 1$ are given numbers, and $\theta(x)$ is the Heaviside step function.

This is analogous to a maximum-entropy problem with a constraint on the cumulative distribution function (CDF) of $p(x)$, and the constraint that $p$ is normalized to unity.

I tried to solve the problem with Lagrange multipliers, but is not clear to me how to do it: I maximize

\begin{equation} \Lambda[p] \equiv - \int_{-\infty}^{+\infty} p(x) \log p(x) \, dx + \lambda \int_{-\infty}^{+\infty}p(x) \left[ \theta(x_{\ast}-x) - q \right]dx + \lambda' \left[ \int_{-\infty}^{+\infty} p(x) \, dx - 1 \right], \end{equation} where $\lambda$ and $\lambda'$ are Lagrange multipliers. I then take the derivative of $\Lambda$ with respect to $p(x)$, set it to zero, and obtain \begin{equation} p(x) = \frac{1}{Z} \exp\left\{ \lambda[\theta(x_{\ast}-x) - q \right]\} \end{equation} which is not normalizable.

Are you aware of where could be the issue? Or whether there is an alternative formulation of this problem which does not have this normalization issue?

Thank you

$\endgroup$
3
  • $\begingroup$ Normalization issue can be overcome if you restrict the support of $p(x)$ to $[0,\infty)$, and enforce $p(x)=0$ for $x<0$. This is the way an exponential distribution gets defined: en.wikipedia.org/wiki/… $\endgroup$ Commented Feb 1, 2017 at 16:06
  • $\begingroup$ This does not solve the normalization issue: the result will still be $p(x) = 1/Z \exp\{ \lambda [\theta(x_{\ast} - x) - q]\}$, which does not converge for large $x$. Note that here $\theta(x)$ is the Heaviside step function. $\endgroup$
    – James
    Commented Feb 1, 2017 at 21:24
  • $\begingroup$ Clearly if you allow the support to be $(-\infty, +\infty)$ then the integral of $p(x)$ is unbounded. So I think the only normalization option is to restrict the support to $[\alpha,\beta]$ such that $\alpha < x^{*} < \beta$, and then equate the sum of two resulting integrals to unity. $\endgroup$ Commented Feb 1, 2017 at 22:18

0

You must log in to answer this question.