Thanks to Szpiro’s/$abc$ conjecture, the answer to your question is true for sufficiently large $N$, up to a factor of $3$ times the primes $p$ (which factor, I believe, can be removed if one is a bit careful in the estimation of the prime factors of $N$ below $3\log m$). Note that the problem you wish to solve with the answer to your question can still be solved (for sufficiently large numbers) if you strengthen your hypothesis to $e^{kp}>N$ for some $k\ge1$ (indeed, $p\gg\log N$ suffices for your problem as $\log n!\gg n$). Here, I’d simply resort to $k=3$ (actually, any $k>9/4=2.25$ suffices) to allow for a simpler proof. As there is no consensus on the truth of Szpiro’s/abc conjecture since the past decade, we may consider this answer conditional.
For a rational elliptic curve $E$ with minimal discriminant $\Delta_E$ and conductor $f_E$, let $$\sigma_E:=\frac{\log|\Delta_E|}{\log f_E}\,.$$ Szpiro’s conjecture states that for every $\varepsilon>0$, there are only finitely many elliptic curves with $\sigma_E\ge 6+\varepsilon$.
Now, consider the rational elliptic curve $$E\colon y^2=x(x-1)(x-m^2)$$ for a given natural number $m$. By well-known/standard results, the above is a minimal model and so the minimal discriminant is $$\Delta_E=(4m^2(m^2-1)^2\,.$$ Note that, in your notation, $N=m^3-m$, and so $\Delta_E=(4mN)^2$. In particular, $E$ has multiplicative reduction at all odd prime factors of $\Delta_E$ but additive reduction at $2$; thus, the conductor $f_E$ is $$f_E=2^{n}\prod_{2<p|N}p\,,$$ for some $n\in\{2,3,4,5,6\}.$ Now, working asymptotically (we write $\sim$ for asymptotically equal to and $\lesssim$ (resp., $\gtrsim$) for asymptotically less (resp., greater) than or equal to), then assuming to the contrary that $e^{kp}\le N$ for $k=3$, or better still some $k>9/4$, we obtain \begin{align} \log f_E&= n\log 2 +\sum_{2<p|N}\log p\\ &\le n\log 2+ \sum_{p\le\frac{1}{k}\log N}\log p\\ &\lesssim n\log 2+ \sum_{p\le\frac{3}{k}\log m}\log p\\ &\sim n\log 2+\frac{3}{k}\log m\,, \end{align} where we have used the prime number theorem $\sum_{p\le x}\log p\sim x$ in the final step. This implies that \begin{align} \sigma_E=\frac{\log|\Delta_E|}{\log f_E}&=\frac{\log 16m^4(m^2-1)}{\log f_E}\\ &\sim\frac{\log 16m^8}{\log f_E}\\ &\gtrsim\frac{4\log2+8\log m}{n\log 2+ \frac{3}{k}\log m}\\ &\approx \frac{8}{3}k>6, \end{align} which contradicts Szpiro’s conjecture for sufficiently large $N$.