It is not true that this equation always has no solutions in the supercritical case $p > \frac{N + 2}{N - 2}$.

The simplest counterexample is on an annulus, say $\Omega = B_R \setminus B_1$: in this case one may search for radial solutions by separating variables, which reduces to solving the second-order ODE
$$
-u'' - \frac{N - 1}{r} u' = u |u|^{p - 1}
$$
with boundary condition $0$ at $R$ and $1$. Here the mountain pass or constrained optimization approaches (applied in 1D) will give the existence of a nonnegative solution, similarly to the ball, so long as $p > 2$. The key point is that $r > 1$, so the coefficients are nonsingular.

The most well-known nonexistence result for this equation is that there are no nontrivial solutions on star-shaped domains. This based on the (Rellich-)Pohozaev identity. Here is a sketch assuming also that the domain is smooth (and, WLOG, star-shaped around the origin):

Set $T = x |\nabla u|^2 - 2 (\nabla u \cdot x) \nabla u - 2 x \frac{|u|^{p+1}}{p+1} $. This is a vector field, and a computation shows that
$$
\text{div} T = (N-2)|\nabla u|^2 - 2 N \frac{|u|^{p + 1}}{p+1}.
$$
At this point one integrates over $\Omega$ and applies the divergence theorem:
$$
\int_\Omega (N-2)|\nabla u|^2 - 2 N \frac{|u|^{p + 1}}{p+1} = \int_{\partial \Omega} T \cdot \nu,
$$
where $\nu$ is the outward unit normal vector to $\partial \Omega$. As $u = 0$ on $\partial \Omega$, we have $\nabla u = \pm |\nabla u| \nu$, so after some simplification
$$
\int_{\partial \Omega} T \cdot \nu = - \int_{\partial \Omega} |\nabla u|^2 x \cdot \nu.
$$
This is a nonpositive quantity as the domain is star-shaped (indeed, with more effort one can check that it is strictly negative for nontrivial solutions).

There is a second, simpler identity available: multiply the PDE by $u$ and integrate by parts:
$$
\int_{\Omega} |\nabla u|^2 = \int_{\Omega} |u|^{p+1}.
$$
Substituting this in,
$$
\int_{\Omega} (N - 2 - \frac{2N}{p + 1})|\nabla u|^2 \leq 0.
$$
When $p > \frac{N + 2}{N - 2}$ the coefficient is positive and this implies that $u$ is constant $0$, a contradiction.

This general approach accounts for a large portion of nonexistence theorems about semilinear equations in the literature.

To answer the question more fully is difficult: generally one expects existence of positive solutions to be related to the topology of the domain. In the critical case ($p = \frac{N + 2}{N - 2}$) this is fairly well understood, see e.g. Bahri, Coron. In the supercritical case this is not understood nearly as well.