Variance of truncated normal distribution

Question

Let $ X \sim \mathcal{N} ( \mu, \sigma^2 ) $, $ - \infty \leqslant a < b \leqslant +\infty $ ($ a, b \ne \infty $ simultaneously) and $ Y $ has a truncated normal distribution on $ (a, b )$, i.e. $ Y $ has pdf

$ \frac{1}{\Phi(\frac{b - \mu}{\sigma}) - \Phi(\frac{a - \mu}{\sigma}) } \frac{1}{\sigma}\phi(\frac{x - \mu}{\sigma}) \mathbf{1} ( a < x < b ) ,$

where $ \phi(x)$ and $ \Phi(x) $ are pdf and cdf of standard normal distribution.

How to prove that the variance of truncated normal distribution is always less than the variance of untruncated distribution, i.e.

$ \mathrm{Var} Y < \mathrm{Var} X $?

Here is a neat proof that $ \mathrm{Var} Y \leqslant \mathrm{Var} X $, but I can't extend it to cover the equality case.

Also there is a proof in this article, but it seems wrong, because they only assumed $ \mu \in (a, b) $.

Update:

Now I think it can be proved in two steps.

1) For one-sided truncations the strick formula $ \mathrm{Var} Y' < \mathrm{Var} X $ follows from estimations of inverse Mills ratio.

2) For two-sided truncations $ \mathrm{Var} Y \leqslant \mathrm{Var} Y' < \mathrm{Var} X $, because one-sided truncated distributions is log-concave.

Is it correct proof or am I missing something?

Answer 1

Since the OP was kind enough to reference an older answer of mine, and also to alert me to the fact, I will provide some input here, of naive flavor, I guess, since mathoverflow is definitely over and out of my league. I will use $\Phi()$ for the standard normal CDF and $\phi()$ for the standard normal PDF. In my answer in math.se I had proved that

$${\rm Var}(Y) = {\rm Var}(X)\cdot \left[1+\sigma^2\frac{\partial^2 \ln H(\mu)}{\partial \mu^2}\right],\;\; -1 <\sigma^2\frac{\partial^2 \ln H(\mu)}{\partial \mu^2} \leq 0 $$

where:

$$\ln H(\mu)=\ln \big(\Phi(\beta(\mu))-\Phi(\alpha(\mu))\big)$$

$$\alpha=(a-\mu)/\sigma, \;\beta=(b-\mu)/\sigma$$.

The weak inequality comes from the fact that $H$ is log-concave (see the original post).

Calculating the second derivative, in order to prove strict inequality, we have to show that

$$\frac{\partial^2 \ln H(\mu)}{\partial \mu^2} < 0 \implies D \equiv [-\phi'(\alpha)+\phi'(\beta)][\Phi(\beta)-\Phi(\alpha)]-[\phi(\alpha)-\phi(\beta)]^2 < 0 $$

We note that $\Phi(\beta)-\Phi(\alpha) >0 $ always.

CASE A : $a \leq \mu \leq b$ (including one-sided trunctaions).

Here, by the unimodality of $\phi()$, with $\mu$ being the mode, we have that

$$\phi'(\alpha) \geq 0, \phi'(\beta) \leq 0 \implies -\phi'(\alpha)+\phi'(\beta) < 0$$

since the two derivatives cannot be both equal to zero.

Then $D < 0$ always (including truncation symmetric around the mean, where, due to the fact that $\phi()$ is an even function, we have that $\phi(\alpha)=\phi(\beta)$, and the second element of $D$ will be zero. But the first element is always strictly negative).
So for this case we have proved that $\partial^2 \ln H(\mu)/\partial \mu^2 < 0$ as we wanted.

The cases left out are all the cases where $\mu$ does not belong to the (two-sided or one-sided) truncated support.

CASE B : Truncated support $S_B\equiv (-\infty , b], \mu \notin S_B$

Here the inequality to prove is

$$D_B \equiv \phi'(\beta)\Phi(\beta)-\phi(\beta)^2 < 0$$

Since $\phi'(\beta) = -\beta \phi(\beta)$ we wan to show

$$-\beta \phi(\beta)\Phi(\beta)-\phi(\beta)^2 < 0 \implies -\beta \Phi(\beta)-\phi(\beta) < 0 \implies \beta \Phi(\beta)+\phi(\beta) > 0$$

But this holds because

$$\beta \Phi(\beta)+\phi(\beta) = \int_{-\infty}^{\beta}\Phi(t){\rm d}t$$

and the integral is necessarily strictly greater than zero since $\Phi()$ is non-negative and non-constantly zero. So we have proved here too what we needed to prove.

CASE C : Truncated support $S_C\equiv [a, \infty ), \mu \notin S_C$

Here the inequality to prove is

$$-\phi'(\alpha)[1-\Phi(\alpha)]-\phi(\alpha)^2 < 0 \implies \alpha \phi(\alpha)[1-\Phi(\alpha)]-\phi(\alpha)^2 < 0 $$

and simplifying and using the symmetry properties of the two functions we have

$$ \implies (-\alpha)\Phi(-\alpha) + \phi(-\alpha) >0$$ and we are at the same situation as in Case B. So QED here too.

I am not treating the two-sided truncation cases $a < b < \mu$ and $\mu < a < b$. As with the cases I treated, I am certain there exists some more advanced and elegant way to prove what we want.

Answer 2

This paper is relevant:

E. Mailhot, Une propriété de la variance de certaines lois de probabilité réelles tronqées, C. R. Acad. Sci. Paris Sér. I Math. 301 (1985) 241–244.

Mailhot proves that the variance of a truncated distribution is monotonic in the truncation point if the (cumulative) distribution function is log-concave, or if it has a continuous density which is log-concave, and also in some discrete cases. Both of the first two conditions are satisfied by the normal distribution.

There is nothing explicitly in the paper about strict increase, but it should follow easily. The difference between the variances for two truncation points is written exactly as the integral of a nonnegative function. All you have to do is calculate the integrand in the normal case and note that it is not identically zero.

If you can't find the paper, send me private mail and I'll send you a copy.