# Tightness of probabilty distributions

Let $\mathcal{P}(\mathbb{N})$ be the set of all probability mass functions on $\mathbb{N}=\{1,2,\dots \}$. Let $E$ be a closed(with respect to pointwise convergence, or equivalently the total variation metric) subset of $\mathcal{P}(\mathbb{N})$ and $Q\notin E$. Let $0<\beta<1$. Now $\displaystyle \sum_{x\in N} P(x)^{\beta}Q(x)^{1-\beta}\le 1$ for any $P$ and $Q$ by Holder's inequality. Let us suppose that $0< s:=\displaystyle \sup_{P\in E}\sum P(x)^{\beta}Q(x)^{1-\beta}$.

Let $\{P_n\}$ be a sequence in $E$ such that $\sum P_n(x)^{\beta}Q(x)^{1-\beta}\to s$

Now my goal is to examine whether $\{P_n\}$ have a convergent subsequence converging to a true probability distribution. By the diagonal argument I can always extract a convergent subsequence, but the limit need not be a probability distribution; it could be a defective probability distribution. But in this problem can one somehow argue that we can extract a convergent subsequence converging to a probability distribution?

I am also thinking of showing that $P_n$ are tight, because in the examples which I have, mass cannot escape to infinity. I hope, if we show tightness, we can extract a convergent subsequence converging to a true probability distribution.

-
Showing tightness is certainly the usual approach to problems of this type. –  Mark Meckes Oct 6 '10 at 14:43
if the subsequence in $E$ you extract converges to a defective distribution, doesn't that contradict the assumption that $E$ is closed? –  ronaf Oct 7 '10 at 1:31
If $E$ were further complete, then it would have been a contradiction. This would be the case if $P(N)$ were complete. But $P(N)$ is not complete, for example, the sequence $\delta_n$(point mass at $n$) converges to $(0,0,\dots)$. –  Ashok Oct 7 '10 at 5:16

## 2 Answers

I think this conjecture is false, that is, there does not necessarily exist a subsequence that converges to a true probability distribution. Consider the following situation:

Let $Q=(1,0,0,0,...)$, i.e. the probability distribution with all mass at $x=1$.

Define the distribution $R_n$, for $n=2,3,...$, as $$R_n(1)=\frac{1}{2} - \frac{1}{n}$$ $$R_n(n)=\frac{1}{2} + \frac{1}{n}$$ and $R_n(x)=0$ for all other values.

Let $E$ be the set of all $R_n$, for $n\geq 2$. Note that $E$ is closed (because any distribution not in $E$ can be separated from it by a sufficiently small $\epsilon$ ball) and that $s=(1/2)^\beta>0$.

Next, we argue that any convergent distributions $P_n$ contained in $E$ can be viewed as a subsequence of the $R_n$. For any $R_n$, $$\sum_{x=1}^\infty R_n^\beta (x) Q^{1-\beta}(x)=\left(\frac{1}{2}-\frac{1}{n}\right)^\beta$$ As $n$ increases, this value increases monotonically to $s$. Therefore, if we have any set of points $P_n$ in $E$ such that $$\lim_{n\rightarrow \infty} \sum P_n^\beta (x) Q^{1-\beta}(x) =s$$ then $P_n$ has a convergent subsequence that is a subsequence of the $R_n$.

Finally, if we consider any subsequence of $R_n$, it does not converge to a probability distribution since half of its probability wafts off to infinity. (A formal proof is straightforward.)

-

it would seem from bill's counterexample that some further constraints on $E$ are needed to get what you want. requiring that $E$ be tight would certainly do the job - but it may be an unnecessarily restrictive assumption.

for example, let $E$ be the [parametric] family of poisson distributions. [so we change the support of the measures to $N := \{0,1,\cdots\}$]. let $\cal P$ denote the set of probability measures on $N$.

for any $Q$, it is pretty clear that as the poisson parameter $\lambda\to\infty$,

$$I_\beta(Poi_\lambda,Q) := \sum_{x\in{N}} Poi_\lambda(x)^\beta Q(x)^{1-\beta}\to 0.$$

so in obtaining $s$, we may restrict attention to some bounded interval $[0,L]$ for $\lambda$. then, as $I_\beta(Poi_\lambda,Q)$ is continuous in $\lambda$, its max is attained on $[0,L]$.

in this example, $E$ is not tight, altho its only pointwise limit points outside itself are $\delta_0$, the measure putting probability 1 at 0, and the zero-measure $\delta_\infty = (0,0,\cdots)$. [$\delta_0$ should be included if one wants $\lambda > 0$ a priori.] here, the only defective limit point is $\delta_\infty$.

additionally: altho it is true [as the OP states] that pointwise convergence [the product topology for $\ell_1$] and "total variation" [or $\ell_1$-norm] convergence are equivalent for $\cal P$, the two are not the same for $\ell_1$. in the poisson example, $\delta_\infty$ is a pointwise limit point but not a strong limit point of $E$. [so $E$ is strongly closed in this case. i think this was involved in my previous (somewhat hastily conceived) comment.]

in bill's example also, $R_\infty$ is also a pointwise but not a strong limit point of $E= \{R_n: n\ge 2\}$.

these examples suggest that when $s>0$, tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution: $\delta_\infty$. [it seems almost obvious then that if $I_\beta(P_n, Q) \uparrow s>0$ as $n\to\infty$, that any convergent subsequence of $\{P_n\}$ must tend to a limit in $\cal P$. a fancier way to put it is that $I_\beta: P \to I_\beta(P,Q)$ is continuous for the $\ell_\infty$-weak topology on $\cal P$, which is really just pointwise convergence by another name in this case.]

an interesting $E$ satisfying this condition is the set of all binomial distributions on $N$, where both $n$ and $p$ are parameters. here $E$ has lots of pointwise limit points it doesn't contain [like the poisson distributions], but only one defective pointwise limit point: $\delta_\infty$.

-
We can't do that in our problem, because we want a probability distribution to attain the supremum. that is, if at all supremum is attained, it should be attained by a probability distribution. Thanks anyway, your discussion was useful. –  Ashok Oct 8 '10 at 12:24
@ashok - can you clarify what you mean by "We can't do that..."? –  ronaf Oct 8 '10 at 15:29
I couldn't understand what you are saying in the following. "...these examples suggest that when $s>0$, tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution: $δ_{\infty}$. [it seems almost obvious then that if $I_β(P_n,Q)\uparrow s>0$ as $n\to \infty$, that any convergent subsequence of ${P_n}$ must tend to a limit in $P$. a fancier way to put it is that $I_β:P\to I_β(P,Q)$ is continuous for the $ℓ_{\infty}$-weak topology on $P$, which is really just pointwise convergence by another name in this case.]..." –  Ashok Oct 9 '10 at 4:28
Especially I couldn't understand the following: "...tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution..." $E$ is a set of PMF's. And our metric space is set of PMF's on $N$ (i.e., $P$ in your notation, $P(N)$ in my notation). So how can the closure of $E$ contain atmost one defective distribution? That is why I said "we can't do that". –  Ashok Oct 9 '10 at 4:36
@ashok- for the poisson family $E$ with $\lambda$ > 0, as $\lambda\to 0$, Poi$_\lambda\to$ [pointwise] $\delta_0$; as $\lambda\to\infty$, Poi$_\lambda\to$ [pointwise] $\delta_\infty$, the null measure [or, if you prefer, the pointmass at $\infty$]. if you think of pointwise convergence in terms of a metric, find the distance between Poi$_\lambda$ and my claimed limits. at any rate this $E$ satisfies the condition that its only defective limitpoint is $\delta_\infty$. [i don't see any other limit points for $E$ that are not already in $E$. do you?] which $E$'s interest you? –  ronaf Oct 9 '10 at 19:09