Since you seem to be mainly interested in $\mathcal{M_g}$, let me suggest a "quick and dirty" approach based on the following fact:

> $\mathcal{M}_g$ is quotient of a nonsingular algebraic variety $\tilde M_g$(and particular complex manifold) by a finite group $G$

Idea: Take $\Gamma(3)\subset \Gamma$ to be the preimage of congruence group $\{M\in Sp_{2g}(\mathbb{Z})\mid M\equiv I \mod 3\}$ under the "Torelli" homomorphism $\Gamma\to Sp_{2g}(\mathbb{Z})$. $\Gamma(3)$ can be shown to act freely on Teichmuller space, so the quotient $\tilde M_g$ is nonsingular with action by $G=Sp_{2g}(\mathbb{Z}/3)$, such that $\mathcal{M}_g$ is the quotient. (It helps to think about the $g=1$ case first.)

This makes a lot of things fairly straightforward. For example, a vector bundle on $\mathcal{M}_g$ can be understood to mean a $G$-equivariant vector bundle of $\tilde M_g$ etc.. [**Added** Since $\tilde M_g$ is a so called fine moduli space, it has a universal family of curves. To this we can associate a Hodge bundle in the usual way. Since it's naturally $G$-equivariant, it yields the Hodge bundle of $\mathcal{M}_g$.]
 More general orbifolds are only locally  quotients, which makes the foundations  more complicated, as people have already explained.