$\DeclareMathOperator\Ker{Ker}$

Let $\mathcal{H}$ be a separable Hilbert space. Then a bounded operator $T$ is **not** in the norm closure of Fredholm operators iff either $T$ or $T^*$ has a finite-dimensional kernel and has a closed image of infinite codimension.

(Note that in particular, the only operators that are in the closure of Fredholm operators but not in the closure of invertible operators, are Fredholm operators of nonzero index.)

*Proof:* Let $T$ not be in the norm closure of Fredholm operators. So it is not in the norm closure of invertible operators. By the Bouldin result mentioned previously, $T$ has a closed image and $\dim(\Ker(T))$ and $\dim(\Ker(T^*))$ differ. If both are finite, $T$ is Fredhlom. So one is finite and the other is infinite.

Conversely, suppose that $T$ satisfies the condition. Up to adjoint, we can suppose that $T$ has closed range, has finite-dimensional kernel and infinite-dimensional cokernel. So there exists a block-decomposition under which we can write $T$ as
$$T=\begin{pmatrix} A_0 & 0 \\ 0 & 0\end{pmatrix},$$
where the right-hand column is finite-dimensional, $A_0$ is invertible, and both rows are infinite-dimensional. Let $T'$ be norm-close to $T$: write it as
$$T'=\begin{pmatrix} A & b \\ C & d\end{pmatrix},$$
so $A$ being close to $A_0$, it is invertible. Suppose by contradiction that $T'$ is Fredholm. So there exists a bounded operator $S$ such that $TS-I$ has finite rank. Write $S=\begin{pmatrix} E & F \\ g & h\end{pmatrix}$ in the transpose block decomposition. (Small letters emphasize the fact that they have either finitely many rows or columns.) Then
$$T'S=\begin{pmatrix} AE+bg & AF+bh \\ CE+dg & CF+dh\end{pmatrix},$$
this time in the symmetric block decomposition corresponding to the row decomposition of $T$ (hence with both blocks infinite-dimensional. Hence $AF+bh$ has finite rank. Since $h$ has finite rank, we deduce that $AF$ has finite rank. Since $A$ is invertible, we deduce that $F$ has finite rank. But then $CF+dh$ has finite rank, which contradicts that $CF+dh-I$ has finite rank.