I'm thinking about the following argument. It's not entirely precise, I will be grateful for comments how to improve it.

Suppose such an $X$ is toric. Because the exceptional curves $E_i$ do not move inside $X$, they are fixed by the torus action. Therefore after blowing them down, the torus still acts and their images are fixed points. But there are only 3 fixed points of the torus action on $\mathbb{P}^2$!

I'm thinking about the following argument. It's not entirely precise, I will be grateful for comments how to improve it.

Suppose such an $X$ is toric. Because the exceptional curves $E_i$ do not move inside $X$, they are fixed by the torus action. Therefore after blowing them down, the torus still acts and their images are fixed points. But there are only 3 fixed points of the torus action on $\mathbb{P}^2$!

Different idea (using fans): since $Pic (X) = \mathbb{Z}^{1+s}$, the fan defining $X$ has $3+s$ rays, and $s$ of them correspond to the exceptional curves $E_1, \ldots, E_s$. Let us denote the torus invariant divisors corresponding to the other 3 rays $D_1, D_2, D_3$. Since $\dim X=2$, these $3+s$ rays/divisors are arranged in a circular order, so that only neighbors on this circle intersect. Because the curves $E_i$ and $E_j$ do not intersect for $i\neq j$, no two $E_i$ are neighbors on the circle, which shows that there at most as many $E$'s are $D's$, that is, $s\leq 3$.

Note that the first argument works more generally for the blow-up of $\mathbb{P}^n$ in $s > n+1$ points.