Question

Let $\mathcal{P}(\mathbb{N})$ be the set of all probability mass functions on $\mathbb{N}={1,2,\dots }$. Let $E$ be a closed(with respect to pointwise convergence, or equivalently the total variation metric) subset of $\mathcal{P}(\mathbb{N})$ and $Q\notin E$. Let $0<\beta<1$. Now $\displaystyle \sum_{x\in N} P(x)^{\beta}Q(x)^{1-\beta}\le 1$ for any $P$ and $Q$ by Holder's inequality. Let us suppose that $0< s:=\displaystyle \sup_{P\in E}\sum P(x)^{\beta}Q(x)^{1-\beta}$.

Let ${P_n}$ be a sequence in $E$ such that $\sum P_n(x)^{\beta}Q(x)^{1-\beta}\to s$

Now my goal is to examine whether ${P_n}$ have a convergent subsequence converging to a true probability distribution. By the diagonal argument I can always extract a convergent subsequence, but the limit need not be a probability distribution; it could be a defective probability distribution. But in this problem can one somehow argue that we can extract a convergent subsequence converging to a probability distribution?

I am also thinking of showing that $P_n$ are tight, because in the examples which I have, mass cannot escape to infinity. I hope, if we show tightness, we can extract a convergent subsequence converging to a true probability distribution.

Answer 1

I think this conjecture is false, that is, there does not necessarily exist a subsequence that converges to a true probability distribution. Consider the following situation:

Let $Q=(1,0,0,0,...)$, i.e. the probability distribution with all mass at $x=1$.

Define the distribution $R_n$, for $n=2,3,...$, as $$R_n(1)=\frac{1}{2} - \frac{1}{n}$$ $$R_n(n)=\frac{1}{2} + \frac{1}{n}$$ and $R_n(x)=0$ for all other values.

Let $E$ be the set of all $R_n$, for $n\geq 2$. Note that $E$ is closed (because any distribution not in $E$ can be separated from it by a sufficiently small $\epsilon$ ball) and that $s=(1/2)^\beta>0$.

Next, we argue that any convergent distributions $P_n$ contained in $E$ can be viewed as a subsequence of the $R_n$. For any $R_n$, $$\sum_{x=1}^\infty R_n^\beta (x) Q^{1-\beta}(x)=\left(\frac{1}{2}-\frac{1}{n}\right)^\beta$$ As $n$ increases, this value increases monotonically to $s$. Therefore, if we have any set of points $P_n$ in $E$ such that $$\lim_{n\rightarrow \infty} \sum P_n^\beta (x) Q^{1-\beta}(x) =s$$ then $P_n$ has a convergent subsequence that is a subsequence of the $R_n$.

Finally, if we consider any subsequence of $R_n$, it does not converge to a probability distribution since half of its probability wafts off to infinity. (A formal proof is straightforward.)

Answer 2

it would seem from bill's counterexample that some further constraints on $E$ are needed to get what you want. requiring that $E$ be tight would certainly do the job - but it may be an unnecessarily restrictive assumption.

for example, let $E$ be the [parametric] family of poisson distributions. [so we change the support of the measures to $N := \{0,1,\cdots\}$]. let $\cal P$ denote the set of probability measures on $N$.

for any $Q$, it is pretty clear that as the poisson parameter $\lambda\to\infty$,

$$I_\beta(Poi_\lambda,Q) := \sum_{x\in{N}} Poi_\lambda(x)^\beta Q(x)^{1-\beta}\to 0.$$

so in obtaining $s$, we may restrict attention to some bounded interval $[0,L]$ for $\lambda$. then, as $I_\beta(Poi_\lambda,Q)$ is continuous in $\lambda$, its max is attained on $[0,L]$.

in this example, $E$ is not tight, altho its only pointwise limit points outside itself are $\delta_0$, the measure putting probability 1 at 0, and the zero-measure $\delta_\infty = (0,0,\cdots)$. [$\delta_0$ should be included if one wants $\lambda > 0$ a priori.] here, the only defective limit point is $\delta_\infty$.

additionally: altho it is true [as the OP states] that pointwise convergence [the product topology for $\ell_1$] and "total variation" [or $\ell_1$-norm] convergence are equivalent for $\cal P$, the two are not the same for $\ell_1$. in the poisson example, $\delta_\infty$ is a pointwise limit point but not a strong limit point of $E$. [so $E$ is strongly closed in this case. i think this was involved in my previous (somewhat hastily conceived) comment.]

in bill's example also, $R_\infty$ is also a pointwise but not a strong limit point of $E= \{R_n: n\ge 2\}$.

these examples suggest that when $s>0$, tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution: $\delta_\infty$. [it seems almost obvious then that if $I_\beta(P_n, Q) \uparrow s>0$ as $n\to\infty$, that any convergent subsequence of $\{P_n\}$ must tend to a limit in $\cal P$. a fancier way to put it is that $I_\beta: P \to I_\beta(P,Q)$ is continuous for the $\ell_\infty$-weak topology on $\cal P$, which is really just pointwise convergence by another name in this case.]

an interesting $E$ satisfying this condition is the set of all binomial distributions on $N$, where both $n$ and $p$ are parameters. here $E$ has lots of pointwise limit points it doesn't contain [like the poisson distributions], but only one defective pointwise limit point: $\delta_\infty$.