That is not an answer.
I want to give an example where the argument of Erdős does not work directly.

Consider an action of group $\Gamma$ on $\mathbb R^3$ generated by the reflections in two lines $x=z=0$ and $x+1=z=0$ and a glide rotation $(x,y,z)\mapsto (-y,x,z+1)$.

The group $\Gamma$ has 2 (up to conjugation) maximal finite subgroups.
(So $2\leqslant 2^3$ — the conjecture is fine.)
Both of these groups isomorphic to $\mathbb Z_2$. 
The factor $X=\mathbb R^3/\Gamma$ is a manifold with two singular circles, say $\Sigma_1$ and $\Sigma_2$.

Let us try to mimic argument of Erdős.
Take subsets $X_i$ of $X$ of midpoints $m$ between $x\in X$ and a closest $x_0\in\Sigma_i$ to $x$.
As in the argument of Erdős we have* $\mathrm{vol}\\, X_i>\tfrac{1}{2^3}\cdot\mathrm{vol}\\, X$.
BUT $X_1\cap X_2$ has interior points and here argument brakes into parts.

**Comment.** 

 (*) Since fixed point sets are 1-dimensional, it would be enough to take $m\in [xx_0]$ such that $\tfrac{|mx_0|}{|xx_0|}=\tfrac1{2\sqrt{2}}$.
But even in this case one has interior points in $X_1\cap X_2$ (the borderline in this example seems to be $\tfrac13$).