Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Let $\mathcal{P}(\mathbb{N})$ be the set of all probability mass functions on $\mathbb{N}=\{1,2,\dots \}$. Let $E$ be a closed(with respect to pointwise convergence, or equivalently the total variation metric) subset of $\mathcal{P}(\mathbb{N})$ and $Q\notin E$. Let $0<\beta<1$. Now $\displaystyle \sum_{x\in N} P(x)^{\beta}Q(x)^{1-\beta}\le 1$ for any $P$ and $Q$ by Holder's inequality. Let us suppose that $0< s:=\displaystyle \sup_{P\in E}\sum P(x)^{\beta}Q(x)^{1-\beta}$.

Let $\{P_n\}$ be a sequence in $E$ such that $\sum P_n(x)^{\beta}Q(x)^{1-\beta}\to s$

Now my goal is to examine whether $\{P_n\}$ have a convergent subsequence converging to a true probability distribution. By the diagonal argument I can always extract a convergent subsequence, but the limit need not be a probability distribution; it could be a defective probability distribution. But in this problem can one somehow argue that we can extract a convergent subsequence converging to a probability distribution?

I am also thinking of showing that $P_n$ are tight, because in the examples which I have, mass cannot escape to infinity. I hope, if we show tightness, we can extract a convergent subsequence converging to a true probability distribution.

share|improve this question
2  
Showing tightness is certainly the usual approach to problems of this type. –  Mark Meckes Oct 6 '10 at 14:43
    
if the subsequence in $E$ you extract converges to a defective distribution, doesn't that contradict the assumption that $E$ is closed? –  ronaf Oct 7 '10 at 1:31
    
If $E$ were further complete, then it would have been a contradiction. This would be the case if $P(N)$ were complete. But $P(N)$ is not complete, for example, the sequence $\delta_n$(point mass at $n$) converges to $(0,0,\dots)$. –  Ashok Oct 7 '10 at 5:16
add comment

2 Answers

up vote 2 down vote accepted

I think this conjecture is false, that is, there does not necessarily exist a subsequence that converges to a true probability distribution. Consider the following situation:

Let $Q=(1,0,0,0,...)$, i.e. the probability distribution with all mass at $x=1$.

Define the distribution $R_n$, for $n=2,3,...$, as $$R_n(1)=\frac{1}{2} - \frac{1}{n}$$ $$R_n(n)=\frac{1}{2} + \frac{1}{n}$$ and $R_n(x)=0$ for all other values.

Let $E$ be the set of all $R_n$, for $n\geq 2$. Note that $E$ is closed (because any distribution not in $E$ can be separated from it by a sufficiently small $\epsilon$ ball) and that $s=(1/2)^\beta>0$.

Next, we argue that any convergent distributions $P_n$ contained in $E$ can be viewed as a subsequence of the $R_n$. For any $R_n$, $$\sum_{x=1}^\infty R_n^\beta (x) Q^{1-\beta}(x)=\left(\frac{1}{2}-\frac{1}{n}\right)^\beta$$ As $n$ increases, this value increases monotonically to $s$. Therefore, if we have any set of points $P_n$ in $E$ such that $$\lim_{n\rightarrow \infty} \sum P_n^\beta (x) Q^{1-\beta}(x) =s$$ then $P_n$ has a convergent subsequence that is a subsequence of the $R_n$.

Finally, if we consider any subsequence of $R_n$, it does not converge to a probability distribution since half of its probability wafts off to infinity. (A formal proof is straightforward.)

share|improve this answer
    
Thanks Bill Bradley, your discussion was useful to me. My actual problem is to find a closed, convex set $E$ s.t. $\sup_{P'\in E'}\sum P'(x)^{\beta}Q'(x)^{1-\beta}$ is not attained, where $P'(x)=\frac{P(x)^{\alpha}}{\sum P(x)^{\alpha}}$, $\alpha=\frac{1}{\beta}$ and $E'={P':P\in E}$. So in your example if we take $E=$Convex hull of ${R_n}$, can we still argue that Supremum over $E'$ is not attained? With $E=$Convex hull of ${R_n}$, I did some simulation. I have a feeling that the supremum over $E'$ is $1$ which should be attained at $(1,0,0,...)$ which is not in $E'$. Am I right? –  Ashok Oct 9 '10 at 12:18
add comment

it would seem from bill's counterexample that some further constraints on $E$ are needed to get what you want. requiring that $E$ be tight would certainly do the job - but it may be an unnecessarily restrictive assumption.

for example, let $E$ be the [parametric] family of poisson distributions. [so we change the support of the measures to $N := \{0,1,\cdots\}$]. let $\cal P$ denote the set of probability measures on $N$.

for any $Q$, it is pretty clear that as the poisson parameter $\lambda\to\infty$,

$$I_\beta(Poi_\lambda,Q) := \sum_{x\in{N}} Poi_\lambda(x)^\beta Q(x)^{1-\beta}\to 0.$$

so in obtaining $s$, we may restrict attention to some bounded interval $[0,L]$ for $\lambda$. then, as $I_\beta(Poi_\lambda,Q)$ is continuous in $\lambda$, its max is attained on $[0,L]$.

in this example, $E$ is not tight, altho its only pointwise limit points outside itself are $\delta_0$, the measure putting probability 1 at 0, and the zero-measure $\delta_\infty = (0,0,\cdots)$. [$\delta_0$ should be included if one wants $\lambda > 0$ a priori.] here, the only defective limit point is $\delta_\infty$.

additionally: altho it is true [as the OP states] that pointwise convergence [the product topology for $\ell_1$] and "total variation" [or $\ell_1$-norm] convergence are equivalent for $\cal P$, the two are not the same for $\ell_1$. in the poisson example, $\delta_\infty$ is a pointwise limit point but not a strong limit point of $E$. [so $E$ is strongly closed in this case. i think this was involved in my previous (somewhat hastily conceived) comment.]

in bill's example also, $R_\infty$ is also a pointwise but not a strong limit point of $E= \{R_n: n\ge 2\}$.

these examples suggest that when $s>0$, tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution: $\delta_\infty$. [it seems almost obvious then that if $I_\beta(P_n, Q) \uparrow s>0$ as $n\to\infty$, that any convergent subsequence of $\{P_n\}$ must tend to a limit in $\cal P$. a fancier way to put it is that $I_\beta: P \to I_\beta(P,Q)$ is continuous for the $\ell_\infty$-weak topology on $\cal P$, which is really just pointwise convergence by another name in this case.]

an interesting $E$ satisfying this condition is the set of all binomial distributions on $N$, where both $n$ and $p$ are parameters. here $E$ has lots of pointwise limit points it doesn't contain [like the poisson distributions], but only one defective pointwise limit point: $\delta_\infty$.

share|improve this answer
    
We can't do that in our problem, because we want a probability distribution to attain the supremum. that is, if at all supremum is attained, it should be attained by a probability distribution. Thanks anyway, your discussion was useful. –  Ashok Oct 8 '10 at 12:24
    
@ashok - can you clarify what you mean by "We can't do that..."? –  ronaf Oct 8 '10 at 15:29
    
I couldn't understand what you are saying in the following. "...these examples suggest that when $s>0$, tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution: $δ_{\infty}$. [it seems almost obvious then that if $I_β(P_n,Q)\uparrow s>0$ as $n\to \infty$, that any convergent subsequence of ${P_n}$ must tend to a limit in $P$. a fancier way to put it is that $I_β:P\to I_β(P,Q)$ is continuous for the $ℓ_{\infty}$-weak topology on $P$, which is really just pointwise convergence by another name in this case.]..." –  Ashok Oct 9 '10 at 4:28
    
Especially I couldn't understand the following: "...tightness for $E$ can be weakened to its pointwise closure having at most one defective distribution..." $E$ is a set of PMF's. And our metric space is set of PMF's on $N$ (i.e., $P$ in your notation, $P(N)$ in my notation). So how can the closure of $E$ contain atmost one defective distribution? That is why I said "we can't do that". –  Ashok Oct 9 '10 at 4:36
    
@ashok- for the poisson family $E$ with $\lambda$ > 0, as $\lambda\to 0$, Poi$_\lambda\to$ [pointwise] $\delta_0$; as $\lambda\to\infty$, Poi$_\lambda\to$ [pointwise] $\delta_\infty$, the null measure [or, if you prefer, the pointmass at $\infty$]. if you think of pointwise convergence in terms of a metric, find the distance between Poi$_\lambda$ and my claimed limits. at any rate this $E$ satisfies the condition that its only defective limitpoint is $\delta_\infty$. [i don't see any other limit points for $E$ that are not already in $E$. do you?] which $E$'s interest you? –  ronaf Oct 9 '10 at 19:09
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.