Let $M$ be a smooth $n$-dimensional manifold, and let $FM = GL(M)$ indicate its tangent frame bundle. Let $G$ be a fixed linear subgroup of $GL(n)$, and consider the space $\mathcal S$ of all $G$-structures on $M$. Each element $Q \in \mathcal S$ is a $G$-subbundle of the frame bundle $FM$. For example, if $G = O(n)$, then $\mathcal S$ gives a parametrization of Riemannian metrics on $M$.
Question 1) Does the space $\mathcal S$ of $G$-structures have a nice structure in its own right? e.g., can one express it in terms of bundles, quotients, etc?
Next, for each $G$-structure $Q \in \mathcal S$, let $\mathcal C_Q$ be a space of principal connections on $FM$ which are compatible with the structure on $Q$.
For example, in the Riemannian case $G = O(n)$, we might focus on torsion-free metric connections, in which case $\mathcal C_Q$ consists of a single point, the Levi-Civita connection for $Q$. We could also focus on more general metric connections, in which case $\mathcal C_Q$ would be non-trivial.
In contexts I like to work in, a geometry consists of some sort of structure like a metric, represented here by the $G$-structure $Q$, and some notion of transport, represented by a choice of a connection $\Gamma \in \mathcal C_Q$. Is there a more standard name for a geometry consisting of a $G$-structure and a connection?
Now, the space of connections is affine, so we may consider the bundle $\pi : \Omega \to \mathcal S$. A point in the bundle $\Omega$ consists of a $G$-structure and a compatible connection, so we may call $\Omega$ the space of geometries on $M$. Again, I would be happy to use a more standard name for this space.
The space $\Omega$ is a fiber bundle, and need not globally decompose into a product $\mathcal S \times \mathcal C$ as is customary in probability theory. Nonetheless, it locally looks a product which should be sufficient for most applications.
Question 2) As in question 1, does the space of geometries $\Omega$ have a nice structure in its own right?
Finally, let's get to probability. Let $\mathcal F$ be the Borel $\sigma$-algebra of $\Omega$. A probability measure $\mathbb P$ over the space $(\Omega, \mathcal F)$ is called the law for a random geometry, or simply a random geometry for short.
My intuition is that a measure $\mathbb P$ is a deterministic object which represents a fuzzy geometry: the picture is that the fuzzy geometry is somehow a superposition of many deterministic realizations of geometries. This intuition can be made more precise. Let $f : \Omega \to \mathcal A$ be some observable of a geometry (measurable function), where $\mathcal A$ is some nice algebra (I'm thinking of the real numbers). Then we may want to take the expectation of $f$, which is simply the integral $\int_\Omega f(\omega) \mathbb P(\mathrm d \omega)$. We don't evaluate $f$ for on any fixed geometry, but instead assign each $\omega$ an infinitesimal weight $\mathbb P(\mathrm d\omega)$ and add up the weighted contributions of $f(\omega)$.
Question 3) Does the space of random geometries $\mathcal P(\Omega)$ have nice geometric structure?
Of course, this space is quite large and there are many such probability measures. We must probably impose additional constraints to do some actual probability theory.
Suppose $M$ is a homogeneous space with symmetry group $\Sigma$ (i.e., $\Sigma$ is a Lie group with a transitive, faithful action on $M$). Of course, when $M$ is equipped with a geometry $\omega \in \Omega$, the space $(M, \omega)$ will not be invariant under the action. This is not a problem, though, since we could naturally impose a symmetry constraint on the law of a random geometry, rather than the geometry itself.
The group $\Sigma$ naturally acts on $\Omega$ in the obvious way: instead of pushing forward a point through a symmetry transformation, we pull back a geometry. This means that $\Sigma$ has a natural action on $\Omega$. We now impose that the law $\mathbb P$ is invariant under the symmetries of the space $M$. That is, we now impose the condition that $\mathbb P = \mathbb P \circ \varphi^{-1}$ for all transformations $\varphi \in \Sigma$.
Let $\mathcal P^\Sigma(M)$ denote the probability laws on $\Omega$ which are invariant under $\Sigma$, and call this the space of symmetric random geometries. Note that a specific realization of a symmetric random geometry need not be symmetric; rather, the law is smeared smoothly over the space $M$.
Question 4) Does the space of symmetric random geometries $\mathcal P^\Sigma(M)$ have nice structure? Are there any simple, natural examples of any symmetric random geometries $\mathbb P \in \mathcal P^\Sigma(M)$?
Thanks for bearing with such a long question. Hopefully, it is accessible to both geometers and probabilists. I thank Ben Bakker, Jarek Korbicz, and Kate Poirier for teaching me some real geometry over the past few weeks, and my apologies to them in advance for the errors which are probably lurking around this post.