Thanks to Szpiro’s/$abc$ conjecture, the answer to your question is true for sufficiently large $N$, up to a factor of $3$ times the primes $p$ (which factor, I believe, can be removed if one is a bit careful in the estimation of the prime factors of $N$ below $3\log m$ in the proof below). Note that the problem you wish to solve with the answer to your question can still be solved (for sufficiently large numbers) if you weaken your hypothesis to $e^{kp}>N$ for some $k\ge1$ (indeed, $p\gg\log N$ suffices for your problem as $\log n!\gg n$). Here, I’d simply resort to $k=3$ (actually, any $k>9/4=2.25$ suffices) to allow for a simpler proof.

For a rational elliptic curve $E$ with minimal discriminant $\Delta_E$ and conductor $f_E$, let
$$\sigma_E:=\frac{\log|\Delta_E|}{\log f_E}\,.$$ Szpiro’s conjecture states that for every $\varepsilon>0$, there are only finitely many elliptic curves with $\sigma_E\ge 6+\varepsilon$.

 
Now, consider the rational elliptic curve
$$E\colon y^2=x(x-1)(x-m^2)$$
for a given natural number $m$. By well-known/standard results, the above is a minimal model and so the minimal discriminant is $$\Delta_E=(4m^2(m^2-1))^2\,.$$
Note that, in your notation, $N=m^3-m$, and so $\Delta_E=(4mN)^2$. In particular, $E$ has multiplicative reduction at all odd prime factors of $\Delta_E$ but additive reduction at $2$; thus, the conductor $f_E$ is $$f_E=2^{n}\prod_{2<p|N}p\,,$$ for some $n\in\{2,3,4,5,6\}.$ Now, working asymptotically (we write $\sim$ for _asymptotically equal to_ and $\lesssim$ (resp., $\gtrsim$) for _asymptotically less (resp., greater) than or equal to_), then assuming to the contrary that $e^{kp}\le N$ for $k=3$, or better still some $k>9/4$, we obtain
\begin{align}
\log f_E&= n\log 2 +\sum_{2<p|N}\log p\\
&\le n\log 2+ \sum_{p\le\frac{1}{k}\log N}\log p\\
&\lesssim n\log 2+ \sum_{p\le\frac{3}{k}\log m}\log p\\
&\sim n\log 2+\frac{3}{k}\log m\,,
\end{align}
where we have used the prime number theorem $\sum_{p\le x}\log p\sim x$ in the final step. This implies that
\begin{align}
\sigma_E=\frac{\log|\Delta_E|}{\log f_E}&=\frac{\log 16m^4(m^2-1)^2}{\log f_E}\\
&\sim\frac{\log 16m^8}{\log f_E}\\
&\gtrsim\frac{4\log2+8\log m}{n\log 2+ \frac{3}{k}\log m}\\
&\sim \frac{8}{3}k>6,
\end{align}
which contradicts Szpiro’s conjecture for sufficiently large $N$.