The usual lower bound (e.g., Theorem II.13 in Davidson and Szarek ("Banach space theory and local operator theory") says that if $A\in R^{n\times d}$ has iid $N(0,1)$ entries then
$$
P(\sigma_{\min}(A^TA)^{1/2} \ge \sqrt n - \sqrt d - t)
\ge 1- e^{-t^2/2}.
$$

You may assume $\sigma_{\min}(\Sigma)>0$ (that is, $\Sigma$ invertible), otherwise there is nothing prove.
Now let $A=X\Sigma^{-1/2}$ so that $A$ has iid $N(0,1)$ entries as above.
Then
$$\sigma_{\min}(X^TX)=\sigma_{\min}(\Sigma^{1/2}A^TA\Sigma^{1/2})
\ge \sigma_{\min}(\Sigma) \sigma_{\min}(A^TA).
$$
Applying the concentration inequality for $\sigma_{\min}(A^TA)$ gives the desired bound.