[Edited to correct errors pointed out by David Ben-Zvi and to answer a query by serge_I. These corrections reduce this ``answer'' to the status of a comment.]

(1) (Over $\mathbb C$) Under strong assumptions on the singularities of $X$ (namely, that $X$ should be cuspidal, Ben-Zvi and Nevins, arXiv 0212094v3), if $\mathcal E$ is coherent on $X$, then giving it a structure as a $\mathcal D_X$-module means giving an isomorphism $\phi:p_1^*\mathcal E\to p_2^*\mathcal E$, where $\mathcal X_1$ is the completion of $X\times X$ along the diagonal and $p_1,p_2:\mathcal X_1\to X$
are the projections. (There is also the cocycle condition,
$p_{31}^*\phi=p_{32}^*\phi\circ p_{21}^*\phi,$
where $p_{ij}:\mathcal X_2\to\mathcal X_1$ are the projections from the completion
$\mathcal X_2$ of $X\times X\times X$ along the diagonal, but we don't need this here.)

Since $X$ is noetherian, there is a unique flattening stratification $X=\coprod X_i$ associated to $\mathcal E$: each $X_i$ is locally closed in $X$ and for any $f:Y\to X$, $f^*\mathcal E$ is locally free if and only if $f$ factors through some $X_i$. The existence
of $\phi$ shows that $p_1^*\mathcal E$ and $p_2^*\mathcal E$ have the same flattening stratification, while $\{p_1^{-1}(X_i)\}_i$ is the flattening stratification for $p_1^*\mathcal E$
and $\{p_2^{-1}(X_i)\}_i$ is the flattening stratification for $p_2^*\mathcal E$.
Since $X$ is irreducible, there is a unique stratum $X_0$ of maximal dimension, so that
$p_1^{-1}(X_0)=p_2^{-1}(X_0)$. Now think in terms of a tubular neighborhood of the diagonal in
$X\times X$ to see that this forces $X_0=X$, so that $\mathcal E$ is locally free provided that $X$ is cuspidal.

(2) (In char. $p$, or over any base) If $X$ is smooth and $\mathcal D_X$ is taken to be the full ring of differential operators rather than the subring generated by those operators of order at most $1$, then the same argument applies.