Suppose that $\mu$ is the law of a Poisson distribution of mean 1, and that $\nu$ is the law of an unknown distribution on the nonnegative integers, though I do know that its mean and variance are both $\lambda$. What, if anything, can be said about the minimum total variation distance between $\mu$ and $\nu$? It would seem natural that the TV distance is minimized when $\nu$ is also a Poisson, but I'm having trouble proving it.
2 Answers
I don't know the answer, but I don't think that Poisson($\lambda$) is best.
What's the word for one minus the total variation distance? i.e. the maximum over all couplings of the probability that two random variables agree? Let's call it the "agreement probability".
The agreement probability between Poisson($1$) and Poisson($\lambda$) decays at least exponentially fast in $\lambda$ (since the probability that Poisson(1) is at least $\lambda/2$ decays faster than exponentially, and the probability that Poisson($\lambda$) is at most $\lambda/2$ decays exponentially).
I think you can do better than this by a distribution that puts more weight at 0. For example, suppose $\lambda$ is an integer, and look at a distribution that puts weight $p$ at $0$, weight $p$ at $2\lambda$, and weight $12p$ at $\lambda$. This has mean $\lambda$ as required, and to get variance $\lambda$ we need
$\lambda = (12p)\lambda^2 + p (2\lambda)^2  \lambda^2$
which gives $p=1/2\lambda$. So for large $\lambda$, the agreement probability with Poisson($1$) is then at least $1/2\lambda$ (because both distributions have weight at least $1/2\lambda$ at $0$).
Anyway, this is just an observation; probably one can do much better than that.

$\begingroup$ Thank you for your response. What if $\lambda$ is between 0 and 1 (in fact, $\lambda$ is very close to 1), so that the weight of $\nu$ at 0 is already greater than the weight of $\mu$ at 0. $\endgroup$ Oct 5, 2012 at 14:06
Maybe you could use Pinsker's Inequality to get an upper bound on the quantity you're interested in ? There are lot of results about finding distributions minimizing the KullbackLeibler divergence (this problem is also sometimes called Information Projection). Though not sure this is useful since you seem more interested in a lower bound...

$\begingroup$ Yes, exactly, I'm only interested in the lower bound. $\endgroup$ Oct 5, 2012 at 14:07