Yes; indeed the number of such $\theta$ can grow as $|X|^{1-2\epsilon}$, which is unbounded for all $\epsilon < 1/2$. For large $N$ let $X$ be the set of integer points $(x,y)$ with $x^2 + y^2 < N$, so $|X| \sim \pi N$. If $l_\theta$ has rational slope $a/b$ then $P_{l_\theta}(x,y)$ depends only on $bx+ay$, which has absolute value at most $\sqrt{a^2+b^2} \cdot N^{1/2}$ by Cauchy-Schwarz. thus $|P_{l_\theta}(X)| \ll \sqrt{a^2+b^2} \cdot N^{1/2}$. This is $\ll |X|^{1-\epsilon}$ if $a^2+b^2 \ll X^{1-2\epsilon}$, and the number of such $\theta$ grows as $X^{1-2\epsilon}$, **QED**.