Ian Morrison wrote up some nice lectures in the book Lectures on Riemann surfaces,World Scientific publishers, Proceedings of the college of Riemann surfaces in 1987, at the ICTM in Trieste.  They were intended as an informal introduction to the two detailed treatments mentioned below by Mumford (l'Enseignement) and Gieseker (Tata).

There is a nice treatment of the chow coordinates of a projective variety in chapter 1 of the book Basic algebraic geometry by Shafarevich.  This is very elementary and readable.

There is a good discussion of the existence of the Hilbert scheme in Mumford's book Lectures on curves on an algebraic surface, Annals of math studies #59.  Sophisticated, but we were able to use it in a seminar long ago, and got some good insight from it.

Mumford (notes by Morrison) first wrote up the case of stable curves in Stability of projective Varieties, in l'Enseignement mathematique, 1977, based on an idea of Gieseker.

Then Gieseker himself presented his version at the Tata Institute in Bombay (TIFR), and wrote it up in their series of lectures on mathematics and physics, #69, 1982.

The original presentation of the concept of stable curves, due to Alan Mayer and David Mumford, is in talks by Mayer and Mumford at the Woods Hole conference 1964, available on James Milne's web site at Michigan, or that of roy smith (mathwonk) at University of Georgia.

As I recall, even the detailed works by Mumford, (GIT, Enseignement), always include some introductory examples and motivation that anyone can read, so one should not shy away from the actual definitive works completely.  In regard to the fine recommendations above, Mukai's is actually a textbook as requested, and not a monograph like most of my recommendations here, but that of course makes it longer.

For beginners, I would observe that the Chow approach is to characterize a projective variety by all lines meeting it, thus getting a subset of the Grassmannian of lines, while the Hilbert approach is to describe a variety by the set of all hypersurfaces of fixed large degree containing it, thus getting a subspace of the vector space of those polynomials, another Grassmannaian.  Then to characterize abstract varieties, one first chooses some natural projective embedding, say by a multiple of the canonical class, then considers the corresponding Hilbert or Chow scheme, and tries to collapse together all different embeddings of the same variety, in GIT by taking a quotient by a group action.  This then leads to singularities at orbits which are smaller than usual, i.e. at points with non trivial isotropy coming from automorphisms of the variety.  These isotropy groups are included in the data of a moduli "stack", but were always considered informative even earlier.

Since the subject is huge, it helps also to know which aspect is of interest.  A moduli space is usually the set of isomorphism classes of objects of a given type.  Hence, aside from foundational subtleties, it “exists” as a set.  Then the problem is to give it more structure and to prove it has some nice properties.  In algebraic geometry one often tries to give it structure as an algebraic space, scheme, or quasi projective variety, perhaps progressively in that order.  So the first job would be to define a natural structure as abstract topological space or even abstract scheme.  Next one wants to capture this structure by some “moduli”.
  
Classically, “moduli” are numbers that distinguish non isomorphic objects, i.e. numerical invariants such as projective coordinates in a field, so this translates into the stage of giving a structure of quasiprojective variety.  This requires finding embedding functions, or sections of line bundles which are constant on equivalence classes.  If the equivalence classes are orbits of a group action, one seeks functions constant on orbits, i.e. “invariants”, and this is the subject of “invariant theory".
  
Since algebraic projective mappings are continuous, their level sets are closed, so the geometric invariant theory problem arises of which orbits are closed.  This leads into various concepts of “stability” of objects under a given action, and also, since closure is a relative notion, of determining certain unstable subsets to exclude so that the remaining orbits become closed.  This is the subject studied by Mumford in which he adapted ideas of Hilbert. 
 
Finally, one wants to find a good geometric compactification of the given moduli space, since the set of isomorphism classes of a given type is seldom compact.  The method of parametrizing moduli spaces by subsets of Hilbert schemes, yields a natural compactification, since Hilbert schemes are projective, but since all isomorphism classes of the original type were already present before compactifying, it is unclear what geometric objects the new points added correspond to.   This leads to the challenge of identifying the Hilbert scheme compactification with a more abstract compactification which adds in degenerate versions of the original geometric objects.  These abstract objects are called perhaps “moduli stable” objects of the orginal kind, and one must show this abstract compact space can be identified with some version of the Hilbert scheme projective compactified one.  
The concept of (moduli) stable curves was introduced by Mayer and Mumford, and the next job was to show they give a good abstract separated compactifiction of M(g).  This is presumably the content of the paper of Deligne and Mumford.  Then the proof they in fact give a natural projective compactification in the Hilbert scheme GIT sense is apparently accomplished in the references of Mumford and Gieseker.

Aside from these global aspects of moduli there are local questions, such as what is the dimension of a (component of a) moduli space, or what is its tangent space?  These are the concern of “deformation theory”, or the local variations of structure of a given object.  Here also one distinguishes deformations of the original objects, usually non singular varieties or manifolds, as in the works of Kodaira, from deformations of the degenerate objects included at the boundary of the compactification, i.e. deformations of singularities.  For the latter there is a nice Tata lecture note by M. Artin, and a recent book by Greuel, Lossen, and Shustin.  All sources rely fundamentally on the unpublished 1964 PhD thesis of M. Schlessinger at Harvard.

After all these foundations are settled, it remains to compute invariant properties of the resulting moduli spaces, their singularities, canonical class, Kodaira dimension, Picard group, cohomology, chow ring, rational curves in them,…..  For M(g)bar this is still going in progress.

However it seems to me most answers, especially mine, are oriented to geometric questions as opposed to the requested arithmetic ones.  Should one suggest some works say by Faltings and Chai?