I am trying to understand how the two paradigms of differential geometry and probability theory can fruitfully be applied to each other.

The more suggestive direction is to use methods of differential geometry to understand the geometric structure of objects in probability theory, e.g. distributions, as done for example in Information Geometry (http://en.wikipedia.org/wiki/Information_geometry).

But what about the other direction? Does it make sense (meaning does one gain more insights) by regarding objects of differential geometry in a probabilistic setup? One could for example consider a distribution of manifolds and instead of deterministic time evolution of a manifold itself, one could investigate the time evolution of random variables that take values in a space of manifolds.

Unfortunately, I didn't find any literature on this second direction. If someone has encountered such problems, I would be thankful for literature suggestions.