# Maximum entropy distribution with constrained quantiles

Suppose we have a continuous probability distribution with density function $f$ whose support is $[a,b]$ and we know that for some finite set of values $\{ v_i \}_{i=1}^n$ between $a$ and $b$ that the $\operatorname{CDF}[f,v_i]=q_i$ where $$a < v_1 \le \cdots \le v_n < b$$ Essentially, we know the CDF of the distribution at various points in the interval, but we don't know it over the entire interval $[a,b]$.

How do we figure out the maximum entropy probability distribution that satisfies these constraints?

Let $$X \sim f$$ which implies that $$X \in [a,b]$$ almost surely. In terms of the CDF of $$X$$ at the points $$\{ v_i \}_{i=1}^n$$, define $$p_i := \mathbb{P}( v_{i-1} < X \le v_i ) = q_i - q_{i-1} \;, \quad 1 \le i \le n+1$$ where we have introduced $$v_0 = a$$, $$v_{n+1} = b$$, $$q_{0}=0$$ and $$q_{n+1} = 1$$. Note that $$\sum_{1 \le i \le n+1} p_i = 1$$ since the sum telescopes.
More to the point, we wish to maximize the entropy $$h(f)$$ subject to the constraints that: $$\int_a^b f(x) dx = 1$$ and $$\int_{v_{i-1}}^{v_i} f(x) dx = p_i$$ for $$1 \le i \le n+1$$. As discussed in the reference below, the density of the maximum entropy distribution which satisfies these constraints is given by: $$f(x) = Z^{-1} \exp\left( \sum_{i=1}^{n+1} \lambda_i 1_{(v_{i-1}, v_i]}(x) \right) 1_{[a,b]}(x)$$ where $$Z$$ is a normalization constant chosen such that $$\int_a^b f(x) =1$$ and where $$\{ \lambda_i \}_{i=1}^{n+1}$$ are Lagrange multipliers chosen such that $$\int_{v_i}^{v_{i+1}} f(x) dx = p_i$$. Eliminating these Lagrange multipliers and writing the density in terms of the given quantiles yields $$f(x) = \sum_i\frac{q_i - q_{i-1}}{v_i - v_{i-1}} 1_{(v_{i-1}, v_i]}(x) \;.$$
• The notation $1_[x,y]$ means a function whose value is 1 on the interval [x,y] and 0 elsewhere? Oct 28, 2016 at 19:07
• (i) Yes, that is exactly what the notation means; sorry for not defining it. (ii) The way to eliminate the $i$th Lagrange multiplier from $f(x)$ is to apply the $i$th constraint $\int_{v_{i-1}}^{v_i} f(x) dx = p_i$ which implies that $Z^{-1} \exp(\lambda_i) (v_i - v_{i-1}) = p_i$. Then use this last equation to eliminate $\exp(\lambda_i)$ from $f(x)$ to obtain the final equation given in the answer. Oct 28, 2016 at 20:14
• Did you mean to write $f(x) = \sum_i \ldots$ in your last equation? Jul 8, 2021 at 7:32