Skip to main content
MathOverflow

Return to Answer

added 8 characters in body
Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $A$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H)$$\mathbb P(X \in A) \le \mathbb P(X \in H^c)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H)$$\mathbb P(Y \in A) \le \mathbb P(Y \in H^c)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t)$$\mathbb P(X \in A) \le \mathbb P(X \in S_t^c)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t)$$\mathbb P(Y \in A) \le \mathbb P(Y \in S_t^c)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $A$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $A$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H^c)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H^c)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t^c)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t^c)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$

deleted 60 characters in body
Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $B$$A$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\quad\quad\quad\quad\quad\Box$$\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$$\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $B$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\quad\quad\quad\quad\quad\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $A$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$

Source Link
dohmatob
dohmatob
  • 6.9k
  • 1
  • 18
  • 76

It turns out that Neyman-Pearson theory helps get a nontrivial inequality.

Notations. For a p.s.d matrix $M$ of size $p$, consider the inner product on $\mathbb R^p$ defined by $\langle x,z \rangle_M := x^TMz$. This induces a norm defined by $\|x\|_M:=\sqrt{\langle x,x \rangle_M}$.

Theorem (Neyman-Pearson for translated multivaritate Gaussians). Let $\beta \in \mathbb R$, $\delta\in \mathbb R^p$ and $B$ be a Borell subset of $\mathbb R^p$. Let $X \sim \mathcal N(0,\Sigma)$ and $Y:=X+\delta$. Consider the half-space \begin{eqnarray} H=\{z \in \mathbb R^p \mid \langle\delta,z\rangle_{\Sigma^{-1}} \le \beta\}. \end{eqnarray}

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in H)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in H)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in H)$.

Proof. The log of the ratio of the densities of $Y$ and $X$ is given by \begin{eqnarray*} \begin{split} \log(f_Y(z))-\log(f_X(z)) &= -\frac{1}{2}(z-\delta)^T\Sigma^{-1}(z-\delta)-\frac{1}{2}z^T\Sigma^{-1}z\\ &= \langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2. \end{split} \end{eqnarray*} Thus $f_Y(z) \le t f_X(z)$ iff $\langle \delta,z \rangle_{\Sigma^{-1}}-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2 \le \log(t)$. Define $t := e^{\beta-\frac{1}{2}\|\delta\|_{\Sigma^{-1}}^2}$. Then $S_t=H$, and we can apply the Neyman-Pearson Lemma (see Appendix below) to get the claimed results. $\quad\quad\quad\quad\quad\Box$

Appendix

The following Lemma is a modern formulation of the celebrated Neyman-Pearson Lemma.

Lemma (Neyman-Pearson 1933). Let $A$ be an event in a probability space $\mathcal Z$, and let $X$, $Y$ be random variables on $\mathcal Z$, with densities $f_X$ and $f_Y$ respectively. Finally, let $t > 0$ and define \begin{eqnarray} S_t:= \{z \in \mathcal Z \mid f_Y(z) \le tf_X(z)\}. \end{eqnarray} We have the following:

  • If $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$.
  • If $\mathbb P(X \in A) \le \mathbb P(X \in S_t)$, then $\mathbb P(Y \in A) \le \mathbb P(Y \in S_t)$.

Proof. Suppose $\mathbb P(X \in A) \ge \mathbb P(X \in S_t)$, and let $A^c$ be the set complement of $A$ in $\mathcal Z$. One computes \begin{eqnarray*} \begin{split} &\mathbb P(Y \in A)-\mathbb P(Y \in S_t)=\int 1_A(z)f_Y(z)dz-\int_{S_t}f_Y(z)dz\\ &= \int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t^c} 1_A(z)f_Y(z)dz-\left(\int_{S_t} 1_A(z)f_Y(z)dz+\int_{S_t} 1_{A^c}(z)f_Y(z)dz \right)\\ &=\int_{S_t^c} 1_A(z)f_Y(z)dz-\int_{S_t} 1_{A^c}(z)f_Y(z)dz\\ &\ge t\left(\int_{S_t^c} 1_A(z)f_X(z)dz-\int_{S_t} 1_{A^c}(z)f_X(z)dz\right),\text{ by definition of }S_t\\ &= t(\mathbb P(X \in A)-\mathbb P(X \in S_t)) \ge 0,\text{ by assumption}. \end{split} \end{eqnarray*} Thus $\mathbb P(Y \in A) \ge \mathbb P(Y \in S_t)$. Similarly, one proves the second part of the claim with "$\ge$" replaced with "$\le$". $\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\Box$