The fraction of the sphere a fixed distance from a subspace

Question

The following problem has a beautiful geometric interpretation in terms of the proportion of points on the Euclidean sphere in $\mathbb{R}^d$ that lie at least a certain distance away from a $k$-dimensional subspace.

Consider a Beta random variable $X$ with shape parameters $k/2$ and $(d−k)/2$, where the parameters $k,d$ are integers that satisfy $0 \lt k \lt d$. What is the best possible upper bound for $\mathbb{P}\{X \lt k/d\}$?

So far, after several days of toil, the best result I have been able to prove is a wretched 0.922. This is based on a nontrivial normal approximation proposed by Alfers & Dinges (1984) and refined by Artstein (2002).

Numerical examples suggest that the extremal case occurs when $k=1$ for every $d \geq 1$. Furthermore, for this case, the limiting value as $d\to\infty$ appears to be equal to $\Phi(1)-\Phi(−1)$, where $\Phi$ is the standard normal cdf.

I can obtain considerably better bounds for this special case, but I have no idea how to prove that these parameters are extremal.

Any thoughts on how to solve these problems?

Answer 1

To start with, your Beta distribution is log-concave for $k,d-k\ge 2$. This already tells you that you have at least $1-1/e\approx 0.632$ on each side of the mean in this case (see, Lovasz, Lemma 5.4), which is attained for $k=2,d\to+\infty$. So, you are left fighting with that stupid $x^{-1/2}$ only. Again, you can generalize to $x^{-1/2}q(x)$ where $q$ is log-concave decreasing on $[0,+\infty)$ (assuming $d\ge 3$; the case $k=1, d=2$ gives $1/2$ by symmetry, so we discard it immediately).

Now let $m$ be the mean. Start with replacing the graph of $\log q$ to the right of $m$ by a straight line preserving the total mass on the right of $m$. The mean will go up because this operation is equivalent to some transport up. Once this is done replace $\log q$ to the left of $m$ by the extension of that line cut at some level to preserve the total mass on the left of $m$. Again, the mean will go up for the same reason. Thus, the only $\log q$ to consider are constant up to some point, linear beyond that point.

It looks like among those the purely linear case is the worst. If it is, indeed, true, the story is over because it is realized as the limiting case $k=1,d\to\infty$. I have no time right now to think of it in honest, but I hope that this helps a bit :)

Answer 2

The uninspiring answer seems to be to copy a formula from Abramowitz & Stegun, Sec. 26.5.21.