2
$\begingroup$

Let $P$ and $Q$ be two probability distributions on $\mathbb{R}$. The goal is to obtain a notion of ``distance'' between $P$ and $Q$, e.g., total variation distance, K-L divergence.

Let $f\colon \mathbb{R} \rightarrow \mathbb{R}$ be a test function. If $f$ satisfies $$ \mathbb{E}_P [ f] = \mathbb{E}_Q[ f],$$ that is, $f$ has the same expectation under $P$ and $Q$. Moreover, if $$ \text{Var}_P[f] = 0~~\text{and}~~ \text{Var}_Q [f] >0,$$ can we say that $TV(P, Q) =1$?

For example, let $P $ be the Rademacher distribution and $Q $ be the standard normal distribution. Consider $f(x) = x^2$. Then $$ \mathbb{E}_P [ f] = \mathbb{E}_Q[ f] =1, ~~\text{Var}_P[f] =0, ~~\text{Var}_Q[f] =1. $$ In this case $TV(P,Q) =1$.

As an extension, if we have $$ \text{Var}_P[f]\leq \epsilon \text{and}~~\text{Var}_Q[f] \geq C $$ for some small number $\epsilon$ and a constant $C$, can we say something about $TV(P,Q)$?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Let $f(x) = x$, $P = \delta_0$, and $Q = (1-2\epsilon)\delta_0 + \epsilon \delta_c + \epsilon \delta_{-c}$. Then $\mathbb{E}_P[f] = \mathbb{E}_Q[f] = 0$ and $\operatorname{Var}_P(f) = 0$. By taking $\epsilon$ small, you can make $TV(P,Q) = 2\epsilon$ as small as desired. By taking $c$ large, you can make $\operatorname{Var}_Q(f) = 2 \epsilon c^2$ as large as desired.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer! If my understand is correct, do you mean when $\text{Var}_P[f] $ is extremely small and $\text{Var}_Q[f] $ is bounded below by a constant, $TV(P, Q)$ can be either large or small? $\endgroup$
    – Steve
    Commented Jan 21, 2017 at 22:07
  • $\begingroup$ @Steve: Yes, correct. You already have an example where it is large. $\endgroup$
    – Nate Eldredge
    Commented Jan 21, 2017 at 22:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .