As already said by Simon in his comment, this is a very vast topic.

Let us stick for simplicity to the case of smooth surfaces $S$ of general type: in this case it is well known that $h^0(S, \, T_S)=0$, hence $\textrm{Aut}(S)$ is a finite group and the moduli functor $\textrm{Def}_S$ is prorepresentable. 

The existence of a quasiprojective coarse moduli space $\mathfrak{M}$ for such surfaces modulo birational equivalence was proven by Gieseker in [**G77**] using GIT theory and the birationality of $5$-canonical map proven by Bombieri in [**B73**]. It follows that for fixed values of $\chi(\mathcal{O}_S)$ and $K_S^2$ the space $\mathfrak{M}_{\chi, \, K^2}$ has a finite number of irreducible components.   

As in the case of $\mathcal{M}_g$, locally in a neighborhood of a point $[S]$ the moduli space $\mathfrak{M}_{\chi, \, K^2}$ is given by a quotient of the base $\textrm{Def}(S)$ of the Kuranishi family of $S$ by the finite group $\textrm{Aut}(S)$. However, in contrast with the case of curves, often pathologies arise, for instence $\textrm{Def}(S)$ can be non reduced. This was first explained by Catanese ([**C89**]) in the case when $K_S$ is not ample, whereas Vakil later proved that such phenomena can be regarded as a particular case of a more general situation that he called the *Murphy's law* for moduli spaces ([**V06**]).

In recent years, many people studied possible compactifications of $\mathfrak{M}_{\chi, \, K^2}$. For instance, Kollar and Shepherd-Barron proposed in [**KSB88**] to add surfaces with semi-log canonical singularities and ample $K_S$, that they called *stable surfaces* by analogy with stable curves. Using this construction, Alexeev and Pardini were for instance able to provide explicit compactifications of the moduli space of Burniat and Campedelli surfaces ([**AP09**]).

Here is a (short) bibliography on the subject:

[**B73**] E. Bombieri: *Canonical models for surfaces of general type*,
Publications Mathématiques de l'IHES **42**, Issue 1 (1973), 171-219. 

[**G77**] D. Gieseker: *Global moduli for surfaces of general tipe*, Invent. math **43** (1977), 233-282.

[**C84**] F. Catanese: *On the moduli spaces of surfaces of general type*, J. Differential geometry **19** (1984), 483-515.

[**C89**] F. Catanese: *Everywhere non reduced moduli spaces*, Invent. math. **98** (1989), 293-310.

[**V06**] R. Vakil: *Murphy's Law in algebraic geometry: Badly-behaved deformation spaces*, Invent. Math. **164** (2006), 569-590. 

[**KSB88**] J. Kollar and N. I- Shepherd-Barron: *Threefolds and deformations of surface singularities*, Invent. Math. **91** (1988), 299-338.

[**AP09**] V. Alexeev, R. Pardini: *Explicit compactifications of moduli spaces of Campedelli and Burniat surfaces*, arXiv:0901.4431.