Below, I provide an answer inspired by the comments of user Terry Tao.
Let $n/N =: \lambda \in (0, 1)$ be the aspect ratio of $A$. We will prove the following.
Claim. For every $C>0$, there exists $c>0$ depending only on $\lambda$ and $C$ such that $s_\min(A) \ge c\sqrt{N}$ w.p $1-2e^{-CN}$.
Technical tools
For any positive integer $k$, let $\mathbb B_k := \{x \in \mathbb R^k \text{ s.t } \|x\| \le 1\}$ be the unit-ball in $k$-dimensional euclidean space $\mathbb R^k$ and let $\mathbb S_{k-1} := \{x \in \mathbb R^k \text{ s.t } \|x\| = 1\}$ be the corresponding unit-sphere.
Fact 1 (Gaussian small-ball probability). If $X = (X_1,\ldots,X_k) \sim \mathcal N(0,I_k)$, then there exists $C_1>0$ such that $P(\|X\| \le u \sqrt{k}) \le (C_1u)^k$ for all $u \ge 0$.
Proof. For every $u \ge 0$, one computes
$$
P(\|X\| \le u \sqrt{k}) = (2\pi)^{-k/2}\int_{u\sqrt{k} \mathbb B_k}e^{-\|x\|^2}dx \le (2\pi)^{-k/2}\mbox{vol}(u\sqrt{k}\mathbb B_k) \le (C_1u)^k,
$$
for some $C_1>0$ (which can be made explicit).
Fact 2 (Spectral-norm upper bound). For every $C>0$, there exists $C_0>0$ such that $s_{\max}(A) \le C_0\sqrt{N}$ w.p $1-e^{-CN}$.
Proof. See Fact 2.4 of this paper by Litvak, Pajor, and Rudelson.
Proof of the main claim
We are now ready to proof the main claim.
Proof of the main claim. Let $\epsilon \in (0, 1)$, to be prescribed later, and let $\mathcal N_\epsilon$ be a maximal $\epsilon$-net for $\mathbb S_{n-1}$. Note that $|\mathcal N_\epsilon| \le (3/\epsilon)^n = (3/\epsilon)^{\lambda N}$. Now, for each $x \in \mathbb S_{n-1}$, there exists $z \in \mathcal N_\epsilon$ such that $\|x-z\| \le \epsilon$. Writing $Ax = Az + A(x -z)$, the triangle inequality gives us $\|Ax\| \ge \|Az\| - \|A(x-z)\| \ge \|Az\|-\epsilon s_\max(A)$. Minimizing both sides, we obtain
$$
s_\min(A) = \inf_{x \in \mathbb S_{n-1}}\|Ax\| \ge \min_{z \in \mathcal N_\epsilon}\|Az\|-\epsilon s_\max(A).\tag{1}
$$
Let $C_1$ be as in Fact 1 with $k=N$. For arbitrary $C>0$, let $C_0$ be as in Fact 2 with $k=N$, and let $c > 0$, to be carefully chosen later. By (1), we know that if $s_\max(A) \le C_0\sqrt{N}$ and $\|Az\| \ge 2c \sqrt{N}$ for all $z \in \mathcal N_\epsilon$, then $s_\min(A) \ge 2c\sqrt{N}-\epsilon C_0\sqrt{N} \ge c\sqrt{N}$ when $\epsilon = c/C_0$ with $c < C_0$. Thus,
$$
\begin{split}
&P(s_\min(A) < c\sqrt{N}) \\
&\quad= P(s_\min(A) < c \sqrt{N},s_\max(A) > C_0\sqrt{N})\\
&\quad\quad\quad+ P(s_\min(A) > C_0\sqrt{N},s_\max(A) \le C_0\sqrt{N})\\
&\quad\le P(s_\max(A) > C_0\sqrt{N}) + P(s_\min(A) < c \sqrt{N},s_\max(A) \le C_0\sqrt{N})\\
&\quad \le e^{-CN} + P(\min_{z \in \mathcal N_\epsilon}\|Az\| < 2c\sqrt{N}) \le e^{-CN} + |\mathcal N_\epsilon|\cdot\max_{z \in \mathcal N_\epsilon}P(\|Az\| \le 2c\sqrt{N})\\
&\quad \le e^{-CN} + (3/\epsilon)^n(C_1 \cdot 2c)^N \le e^{-CN}+((3/\epsilon)^\lambda\cdot C_1\cdot 2c)^N\\
&\quad\le e^{-CN} + (2C_1(3C_0/c)^\lambda c)^N \le e^{-CN} + e^{-CN}=2e^{-C N},
\end{split}
$$
for sufficiently small $c \in (0,C_0)$ such that $2C_1(3C_0/c)^\lambda c < e^{-C}$.
Therefore, given arbitrary $C>0$, the bound $P(s_\min(A) \le c\sqrt{N}) \le 2e^{-CN}$ is guaranteed by taking $c \in (0,c_\lambda(C))$, where
$$
\begin{split}
c_\lambda(C) := \min(C_0,(2C_1e^C(3C_0)^\lambda)^{\frac{-1}{1-\lambda}})>0.
\end{split}
\tag{2}
$$
This completes the proof of the claim. $\quad\quad\Box$
Note. An important highlight the above proof is that it works for every aspect ratio $\lambda \in (0,1)$.
Going beyond Gaussian matrices
A careful inspection of the proof of main ingredients Fact 1 and Fact 2, reveals that we can replace the base distribution $N(0,1)$ of the coefficients of $A$ by any symmetric unit-variance $\sigma^2$-subGaussian such that $0 \le \sigma \le 1$.
