When is differential geometry on moduli spaces possible (and productive)? I will start out by saying I am not well-informed about moduli theory in the slightest. However, it is known that some moduli spaces (of algebro-geometric objects) have severe pathologies (Ex: Murphy's Law with regards to Hilbert schemes), while others are very nice, such as $\overline{\mathcal{M}}_{1,1}$ which can be identified with the famous modular surface, and on such things differential geometry can be done. This leads me to ask (whether anyone has already asked):
1) When does a moduli space (parametrizing algebro-geometric objects) admit natural Riemannian structure(s) besides possibly on some singular locus?
2) In this case, do geodesics correspond to useful information (nice degenerations between objects perhaps)?
 A: You need that the objects are not varying `wildly'. In algebraic geometry, this may mean that they can always be fit into flat families. Also you might find that the moduli space is broken up into many components: this is the case if considering all Riemann surfaces (of arbitrary genus) or all sheaves on a scheme. (Actually the same underlying issue here is that the Hilbert polynomial is misbehaving.) If this is happening, restrict your class of objects a bit. 
If you are in a nice situation, say with a single connected component of moduli, then one tries to exhibit the base objects as local patches of the moduli space. If some of the objects you are parameterizing happen to be very symmetric, i.e., admit larger automorphism groups than their friends nearby, then the structure of the moduli space at this point will have a `stacky' behaviour. Two ways of dealing with this:


*

*Rigidify: For example, on curves (Riemann surfaces), one can assume as part of the data sets of points $S = \{p_0, \ldots, p_n\}$. Or, perhaps, as is done in number theory, choose a basis of the torsion group $E[p]$, for an elliptic curve $E$

*Admit defeat and work with stacks. Particularly nice stacks are $DM$-stacks and arose specifically for dealing with moduli of curves. If you are interested in the Riemannian angle, then it should be pointed out that putting Riemannian metrics on orbifolds (a type of stack, without a lot of AG baggage if you want) is totally a thing.


So for your question:


*

*When does a moduli space admit a Riemannian structure? Ans: This happens when the objects of interest happen to always live in families $E \to B$ where the base $B$ has some natural Riemannian structure.

*In this case, do geodesics correspond to useful information? Ans: This is really up to how you set up moduli problem. And there will in general be multiple ways of assigning meaningful metrics, e.g., there is an abundance of meaningful metrics on $M_g$ (not all Riemannian). What do geodesics mean? Maybe quickest way to deform one object into the other given some constraints.
A: Check out Curt McMullen's paper, and references thereto.
McMullen, Curtis T., The moduli space of Riemann surfaces is K\"ahler hyperbolic, Ann. Math. (2) 151, No.1, 327-357 (2000). ZBL0988.32012.
