Is there a systematic theory for Gibbs measures (better if on Hilbert spaces)?

Question

During these first months in my PhD, I realized how my computational problems can be drastically reduced to one single problem:

Find an efficient way to sample from a Gibbs measure.

Let me elaborate: if $H$ is a Hilbert space, $\mu$ a gaussian measure on it, then I need to numerically approximate a probability measure of the form $\mu_1(x) = Z^{-1} e^{-G(x)} \mu(dx)$ for a "potential" $G$ and a normalization constant $Z$.

I do not have a fixed class for $G$, sometimes $H$ is just $\mathbb{R}^d$ for $d >> 1$, and generally the hypothesis on it might vary a lot (say, $G$ might come from a Neural Network, or from a PDE discretization).

It would be absolutely fantastic to study how the properties on $G$ afflict $\mu_1$, and how they might be exploited e.g. to project numerical methods (in my case, e.g., a good approximation of $\mu_1$ might be used for image reconstruction).

By searching on the web I realized how they are commonly called "Gibbs measure". I was very excited and tried to understand more, but on the other hand I found "only" material concerning the discrete case (lattices). This is certainly a starting point, but I was a bit confused and the approaches were different. Therefore I ask:

Does a systematic theory for the (generic) problem above exist? Are there classics books that you recommend to study? Can you suggest some literature / papers?

(Theoretical investigations are welcome, as well as numerics-oriented results)

Thanks in advance.

Ps: my background is in mathematics, but I am in a Computer Science department. People around me tried to help, and my question arouse after having successfully discussed with them - we are all very curious!

Answer 1

Any probability measure $\mu_1$ absolutely continuous with respect to $\mu_1$ can be written as a Gibbs measure if you allow $G$ to take values $\pm \infty$. If the density is bounded above and below, $G$ will be bounded. So you're basically asking about how to sample from a probability measure. This is a big field of study.
Markov chain Monte Carlo (MCMC) methods are commonly used, but they may run into difficulties, especially when the energy landscape has deep valleys separated by high barriers.

Answer 2

The field of improving convergence of sample averages is known as "enhanced sampling". As Robert pointed out, this is an incredibly hard problem. In my field (theoretical chemistry), we have been struggling with it for the last-half century.

The correct way to approach this problem on depends heavily on what information you have available to you. The simplest case is when you can evaluate $Z$ exactly. In this case, your best bet is likely survey sampling. Failing that, let's assume you can evaluate $G$ at all points $x$ at reasonable cost. If:

1. you have a good idea of what $\mu$ looks like "globally" (i.e. you know where in $H$ its modes are located, their variances, and the decay of the tails as you move away from $G$) your best is likely to be importance sampling.
2. you don't know what $\mu$ looks like, but you know it is dominated by movement along a known lower-dimensional manifold, you can try umbrella sampling or metadynamics.
3. you can evaluate $G$ at all points $x$ at reasonable cost, and you know that $G$ increases reasonably gently, you can try parallel tempering (this is basically the same as umbrella sampling, mathematically, just applied in a different way).
4. you can evaluate $G$ at all points $X$, and you have samples from a related distribution $\pi$ that you don't understand well, but is easy to sample, you can try annealed importance sampling or Hamiltonian replica exchange.
5. evaluation of $G$ is possible but expensive, and the problem isn't too high-dimensional, you can look into surrogate modeling.
6. you can't directly evaluate $G$, but you have a dynamics that preserves $\mu$, you can try Nonequilibrium Umbrella Sampling.

There are many other options and algorithms and this a very much an active area of research. However, hopefully this is enough to point you in the right direction. If you are looking for a mathematical treatment of the subject, "Free Energy Computations: A Mathematical Perspective" could be a good start. It's a bit hard to recommend reading without knowing more about the type of problem you have at hand. Nevertheless, this should hopefully be a good start.