Over the complex numbers, you might be interested in: Mostafa:Die Singularitäten der Modulmannigfaltigkeit $\overline M_g(n)$ der stabilen Kurven vom Geschlecht $g\geq 2$ mit $n$-Teilungspunktstruktur. (German) [The singularities of the moduli variety $\overline M_{g}(n)$ of stable curves of genus $g\geq 2$ with $n$-division point structure] J. Reine Angew. Math. 343 (1983), 81–98.
Over a field of any characteristic, in my paper § 3, the scheme $\overline M_{2}$ over $\mathbb Z$ (and over any field $k$) is described as the normalization of the blowup of the weighted projective scheme $$\mathrm{Proj}\mathbb Z[J_2, J_4, J_6, J_8, J_{10}]/(J_4^2-J_2J_6+4J_8)$$ (the $J_i$'s are Igusa invariants and have weight $i$) along some explicit center. The singularities over $k$ are described as well.
