5
$\begingroup$

Suppose $X_1, X_2, X_3 \sim N(0, 1)$ are three independent standard normal random variables. I am interested in showing that:

$$\text{Var}[X_2\mid X_2 \geq X_1 - a, X_1 \leq X_3 + b] < 1,$$

where the inequality is strict. I've run some simulations and this seems to be true for various choices of $a$ and $b$, and it's also intuitively reasonable, since for any fixed value of $X_1$, this is just the variance of a normal random variable truncated to $(-\infty, X_1 - a)$. I've tried both using the law of total variance (to no avail) and also reparameterizing in terms of $[X_2, X_1 - X_2, X_1 - X_3]$ and using the closed form for the variance of a truncated multivariate normal (Eq. 16), also to no avail.

I'm happy with a loose bound as long as I can show it's strictly less than one!

$\endgroup$
6
  • 1
    $\begingroup$ (i) $X_2$ is independent of $(X_1,X_3)$. So, the condition $X_1 \le X_3 + b$ can be omitted. (Or is there a typo in your post?) (ii) Are any conditions on $a$ and $b$, or at least on their signs, imposed? $\endgroup$ Jun 21, 2023 at 20:24
  • 1
    $\begingroup$ @IosifPinelis $X_2$ is independent of $X_1$ and $X_3$, but it feels like there is a weird dependence between them introduced by the inequalities? For example, cases where $X_1 >> b$ and $X_2 >> X_1$ are very unlikely under both inequalities, but become more likely under just the one. It's possible I am making a logical error though. $\endgroup$
    – B Merlot
    Jun 21, 2023 at 21:25
  • $\begingroup$ @IosifPinelis For (ii) there is currently no bound on $a$ and $b$ or their sign, but I'd be happy with, e.g., something that says that they must both lay in some interval $[-C, C]$. $\endgroup$
    – B Merlot
    Jun 21, 2023 at 21:27
  • $\begingroup$ @BMerlot : You don't need to write $X_2>>X_1;$ you can write $X_2 \gg X_1,$ coded as X_2\gg X_1. MathJax is as good at some things as LaTeX is, and LaTeX is very very good. Just enter "LaTeX symbols" into a search engine and you can find things. $\endgroup$ Jun 22, 2023 at 18:24
  • $\begingroup$ Not only does Iosif Pinelis have a point but omission of the condition $X_1\le X_3+b$ will make Monte Carlo simulations run faster. $\endgroup$ Jun 22, 2023 at 18:32

1 Answer 1

4
+50
$\begingroup$

The inequality is a special case of the following claim.

Claim: If $X = (X_1, \dotsc, X_d) \sim N(\mu, \Sigma)$ is an $\mathbb{R}^d$-valued normal random variable with invertible covariance matrix $\Sigma$, $L : \mathbb{R}^d \to \mathbb{R}$ is linear, and $K \subseteq \mathbb{R}^d$ is convex and has a nonempty interior, then $\operatorname{Var}(L(X) \mid X \in K) \le \operatorname{Var}(L(X))$, with equality if and only if $L(X)$ and $I(X \in K)$ are independent. (Here $I(X \in K)$ is the indicator function of the event $X \in K$.)

Proof: We may assume that $L \neq 0$. After a linear change of coordinates, we may assume that $X$ is standard normal and $L(X) = X_1$. Then the PDF of $X_1$ is $\frac{1}{\sqrt{2 \pi}} e^{-\frac{x^2}{2}}$, and the PDF of $Z = (X_1 \mid X \in K)$ is $\varphi(x) = C e^{-\frac{x^2}{2}} P(X' \in K_x)$, where $X' = (X_2, \dotsc, X_d)$, $$ K_x = \{(x_2, \dotsc, x_d) \in \mathbb{R}^{d-1}; \, (x, x_2, \dotsc, x_d) \in K\} $$ and $C > 0$. The function $\psi \colon \mathbb{R} \to [0,1]$, $x \mapsto P(X' \in K_x)$ is log-concave: if $x, y \in \mathbb{R}$ and $\lambda \in (0,1)$, then $$ \psi((1-\lambda) x + \lambda y) \ge P(X' \in (1-\lambda) K_x + \lambda K_y) \ge \psi(x)^{1-\lambda} \psi(y)^{\lambda}, $$ because normal distributions are log-concave (see https://en.wikipedia.org/wiki/Logarithmically_concave_measure). Note that $X_1$ and $I(X \in K)$ are independent if and only if $\psi$ is a constant function. The Claim follows from the following lemma.

Lemma: If $Z$ is a real random variable with PDF $\varphi(x)$ such that $e^{\frac{x^2}{2}} \varphi(x)$ is log-concave, then $\operatorname{Var}(Z) \le 1$, with equality if and only if $Z \sim N(E[Z],1)$.

Proof: We may add a constant to $Z$, so we may assume that $\varphi$ is maximal at $0$. Then $\varphi$ is decreasing in $\mathbb{R}_{\ge 0}$, increasing in $\mathbb{R}_{\le 0}$, and $\varphi(x) \le \varphi(0) e^{-\frac{x^2}{2}}$ for every $x \in \mathbb{R}$. We have \begin{align} & 1 - \operatorname{Var}(Z) \ge \int_{\mathbb{R}} (1-x^2) \varphi(x) \, dx \\[6pt] = {} & \int_0^\infty (1-x^2) \varphi(x) \, dx + \int_0^\infty (1-x^2) \varphi(-x) \, dx. \end{align} We prove that $\int_0^\infty (1-x^2) \varphi(x) \, dx \ge 0$. This is clear if $\varphi(1) = 0$, so let $\varphi(1) > 0$. There is a $c \ge 0$ such that $\tilde{\varphi}(1) = \varphi(1)$, where $\tilde{\varphi}(x) = \varphi(0) e^{-c x - \frac{x^2}{2}}$. Then $\varphi(x) \ge \tilde{\varphi}(x)$ for $x \in [0,1]$, and $\varphi(x) \le \tilde{\varphi}(x)$ for $x \in [1,\infty)$, so $\int_0^\infty (1-x^2) \varphi(x) \, dx \ge \int_0^\infty (1-x^2) \tilde{\varphi}(x) \, dx = \varphi(0) \int_0^\infty (1-x^2) e^{-c x - \frac{x^2}{2}} \, dx$. We can do the same for $\int_0^\infty (1-x^2) \varphi(-x) \, dx$. So all we need now is that $$ \int_0^\infty (1-x^2) e^{-c x - \frac{x^2}{2}} \, dx > 0 $$ for $c > 0$. We have $\int_0^\infty e^{-c x - \frac{x^2}{2}} \, dx = \frac{\sqrt{\pi}}{\sqrt{2}} e^{\frac{c^2}{2}} \operatorname{erfc}(\frac{c}{\sqrt{2}})$, and differentiating twice in $c$ we get $\int_0^\infty x^2 e^{-c x - \frac{x^2}{2}} \, dx$. In the end, the inequality boils down to $c e^{\frac{c^2}{2}} \operatorname{erfc}(\frac{c}{\sqrt{2}}) < \frac{\sqrt{2}}{\sqrt{\pi}}$. Equivalently, $\frac{\sqrt{\pi}}{2} \operatorname{erfc}(x) < x^{-1} e^{-x^2}$ for $x > 0$. This follows from $(\frac{\sqrt{\pi}}{2} \operatorname{erfc}(x) - x^{-1} e^{-x^2})' = (1+x^{-2}) e^{-x^2} > 0$, since the limits at $x \to \infty$ are $0$.

$\endgroup$
9
  • $\begingroup$ How did you get $\phi(x) \ge \tilde{\phi}(x)$ for $x \in [0,1]$ and $\phi(x) \le \tilde{\phi}(x)$ for $x \in [1,\infty)$? Using the log-concavity of $\phi$, I can get $\phi(x)\ge\phi(0)e^{-cx-x/2}$ for $x\in[0,1]$ and $\phi(x)\le\phi(0)e^{-cx-x/2}$ for $x\in[1,\infty)$ -- but this is not quite what is needed. $\endgroup$ Jun 28, 2023 at 15:28
  • $\begingroup$ @IosifPinelis $\phi(x) / \widetilde{\phi}(x)$ is log-concave, and it is $1$ at both $x = 0$ and at $x = 1$, so it is $\ge 1$ for $x \in [0,1]$, and $\le 1$ for $x \in [1,\infty)$. $\endgroup$
    – user42355
    Jun 28, 2023 at 15:35
  • $\begingroup$ I see. I forgot about the condition that $e^{x^2/2} \phi(x)$ is log concave, not (just) $\phi(x)$ is log concave. Very nice answer! $\endgroup$ Jun 28, 2023 at 15:41
  • $\begingroup$ Concerning the latter displayed integral in your answer being $>0$, you can just note that it equals $c(1-cr(c))$ and use the well-known inequality $cr(c)<1$, where $r$ is the Mills ratio for the standard normal distribution. $\endgroup$ Jun 28, 2023 at 16:06
  • 1
    $\begingroup$ @BMerlot We can transform X to standard normal first. Then we can use an orthogonal transformation, and get $L = a X_1$ for some nonzero $a$ (note that if $X$ is standard normal, and $A$ is an orthogonal matrix, then $A X$ is also standard normal). Finally, multiplying $L$ by a nonzero constant we can get $L = X_1$. $\endgroup$
    – user42355
    Jun 29, 2023 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.