Question

Suppose that $\mu$ is the law of a Poisson distribution of mean 1, and that $\nu$ is the law of an unknown distribution on the non-negative integers, though I do know that its mean and variance are both $\lambda$. What, if anything, can be said about the minimum total variation distance between $\mu$ and $\nu$? It would seem natural that the TV distance is minimized when $\nu$ is also a Poisson, but I'm having trouble proving it.

Answer 1

I don't know the answer, but I don't think that Poisson($\lambda$) is best.

What's the word for one minus the total variation distance? i.e. the maximum over all couplings of the probability that two random variables agree? Let's call it the "agreement probability".

The agreement probability between Poisson($1$) and Poisson($\lambda$) decays at least exponentially fast in $\lambda$ (since the probability that Poisson(1) is at least $\lambda/2$ decays faster than exponentially, and the probability that Poisson($\lambda$) is at most $\lambda/2$ decays exponentially).

I think you can do better than this by a distribution that puts more weight at 0. For example, suppose $\lambda$ is an integer, and look at a distribution that puts weight $p$ at $0$, weight $p$ at $2\lambda$, and weight $1-2p$ at $\lambda$. This has mean $\lambda$ as required, and to get variance $\lambda$ we need

$\lambda = (1-2p)\lambda^2 + p (2\lambda)^2 - \lambda^2$

which gives $p=1/2\lambda$. So for large $\lambda$, the agreement probability with Poisson($1$) is then at least $1/2\lambda$ (because both distributions have weight at least $1/2\lambda$ at $0$).

Anyway, this is just an observation; probably one can do much better than that.

Answer 2

Maybe you could use Pinsker's Inequality to get an upper bound on the quantity you're interested in ? There are lot of results about finding distributions minimizing the Kullback-Leibler divergence (this problem is also sometimes called Information Projection). Though not sure this is useful since you seem more interested in a lower bound...