The following problem has a beautiful geometric interpretation in terms of the proportion of points on the Euclidean sphere in $\mathbb{R}^d$ that lie at least a certain distance away from a $k$-dimensional subspace.

Consider a Beta random variable $X$ with shape parameters $k/2$ and $(d−k)/2$, where the parameters $k,d$ are integers that satisfy $0 \lt k \lt d$. What is the best possible upper bound for $\mathbb{P}\{X \lt k/d\}$?

So far, after several days of toil, the best result I have been able to prove is a wretched 0.922. This is based on a nontrivial normal approximation proposed by Alfers & Dinges (1984) and refined by Artstein (2002).

Numerical examples suggest that the extremal case occurs when $k=1$ for every $d \geq 1$. Furthermore, for this case, the limiting value as $d\to\infty$ appears to be equal to $\Phi(1)-\Phi(−1)$, where $\Phi$ is the standard normal cdf.

I can obtain considerably better bounds for this special case, but I have no idea how to prove that these parameters are extremal.

Any thoughts on how to solve these problems?

  • $\begingroup$ I took the liberty of fixing some broken formatting $\endgroup$ – Yemon Choi Feb 27 '13 at 5:43
  • $\begingroup$ If you changed your title to reflect your geometric interpretation I suspect you'd get more traffic. $\endgroup$ – R Hahn Feb 27 '13 at 16:42
  • 1
    $\begingroup$ Shiri Artstein's "Proportional Concentration Phenomena on the Sphere" (tau.ac.il/~shiri/israelj/ISRAJ.pdf ) might be relevant for the geometric interpretation. $\endgroup$ – Kevin P. Costello Feb 27 '13 at 22:06
  • $\begingroup$ Thanks. That's the paper that I was referring to... So far as I can tell, it focuses on bounds for the probability away from the level $k/d$. $\endgroup$ – jat Feb 27 '13 at 23:30
  • $\begingroup$ Are you looking for a uniform bound in $k$ and $d$? Otherwise, can you clarify what do you mean by a "best possible bound" (that wouldn't yield a tautological answer)? $\endgroup$ – cardinal Apr 29 '13 at 2:02

To start with, your Beta distribution is log-concave for $k,d-k\ge 2$. This already tells you that you have at least $1-1/e\approx 0.632$ on each side of the mean in this case (see, Lovasz, Lemma 5.4), which is attained for $k=2,d\to+\infty$. So, you are left fighting with that stupid $x^{-1/2}$ only. Again, you can generalize to $x^{-1/2}q(x)$ where $q$ is log-concave decreasing on $[0,+\infty)$ (assuming $d\ge 3$; the case $k=1, d=2$ gives $1/2$ by symmetry, so we discard it immediately).

Now let $m$ be the mean. Start with replacing the graph of $\log q$ to the right of $m$ by a straight line preserving the total mass on the right of $m$. The mean will go up because this operation is equivalent to some transport up. Once this is done replace $\log q$ to the left of $m$ by the extension of that line cut at some level to preserve the total mass on the left of $m$. Again, the mean will go up for the same reason. Thus, the only $\log q$ to consider are constant up to some point, linear beyond that point.

It looks like among those the purely linear case is the worst. If it is, indeed, true, the story is over because it is realized as the limiting case $k=1,d\to\infty$. I have no time right now to think of it in honest, but I hope that this helps a bit :)


The uninspiring answer seems to be to copy a formula from Abramowitz & Stegun, Sec. 26.5.21.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.