Projections in infinite dimensional statistical manifolds

Question

I'm struggling to understand the geometry of projection for infinite dimensional statistical manifolds. In finite dimensions, a strictly convex smooth function $F$ defines a Bregman divergence. From this function/divergence, an finite-dimensional information-geometric structure is defined, which is dually-flat. The Riemannian metric is obtained from the Hessian of the strictly convex function, and the rest of the information is obtained from partial derivatives (including the connection $\nabla^{F}$). In fact, there is a global coordinate system $[\theta]$ defined by the gradient $\nabla F$. In this case, it can be shown that unique projections from a point on the manifold to a submanifold exist, provided the submanifold is $\nabla$-flat (i.e. corresponds to a $\nabla$-affine subspace in the $[\theta]$-coordinate system). This, of course, broadly generalizes a projection to an affine subspace of Euclidean space. Does there exist a suitable analog for these facts in the infinite-dimensional information case?

Answer 1

Are you trying to construct an infinite-dimensional (Hilbert) manifold of probability measures on a fixed measurable space? Or you simply want to find a Bergman divergence analog in order to generalize a divergence-like metric in an infinite-dimensional manifold? It is not entirely clear in the OP what kind of analog you are looking for.

In a finite-dimensional statistical manifold, for example an exponential family with natural parameterizations, the manifold does come with a canonical connection as you mentioned. In the finite-dimensional situation, since we can take a parameterization, the geometry of the statistical manifold is embedded in the parameter space. The image of such a projection to a $\nabla$-flat submanifold would correspond to a sub-family of the exponential family, also a restricted collection of parameters that define this sub-family.

In an infinite-dimensional statistical manifold, assuming that you want to compare two points; you can simply calculate the norms of their difference and other kind of similarity measures are possible [Harandi et.al.]. If you are talking about infinite-dimensional statistical manifold that consists of probability measures that cannot be The inner product structure like we have for low-dimensional parameter space, however, does not always exist. In a special case pointed out by [Newton2], we can use an $\alpha$-divergence (which is a generalization to Bergman divergence, see this post from stat.SE) to partially describe a similar inner product-like structure for an infinite-dimensional statistical manifold. The asymptotic results from geometric perspective [Kass&Vos] may generalize into infinite-dimensional situation if you can find a "good finite-dimensional basis approximation". However, a strict inner product structure is not always possible in infinite-dimensional situation, therefore projections may not generalize. If it does generalize, its meaning would likely be a sub-family of probability measures.

In another direction, if you wish to construct a realization (or draw samples) from an inifinite-dimensional statistical manifold, the starting point would be Dirichlet processes wiki. And [Newton] proposed to use transformation (Fenchel–Legendre transform) approach to construct new infinite-dimensional statistical manifolds in a more abstract way.

Reference

[Newton2] Infinite-dimensional statistical manifolds based on a balanced chart, 2016.

[Harandi et.al.] Bregman Divergences for Infinite Dimensional Covariance Matrices,2014.

[Newton] An infinite-dimensional statistical manifold modelled on Hilbert space,2012.

[Kass&Vos] Geometrical Foundations of Asymptotic Inference, 1997.

Answer 2

This is not a direct answer to your question, but rather a pointer to some literature. The book "Information Geometry" by Ay et al. does generalize a bunch of existing machinery from the finite-dimensional setting to an infinite-dimensional one. From a quick skim through the book, they do not address your question directly, but it's plausible that enough other general facts about infinite dimensional statistical models are proved therein to say something towards your question. Good luck!