5
$\begingroup$

Given a nice real valued functional $C$ on some probability space $(\Omega, \mathcal F, P_0)$ we have the following Donsker-Varadhan variational representation

$$\log E_{P_0}\left[e^C\right]=\sup_{P\sim P_0}\{E_P[C]-D_{KL}(P||P_0)\},$$

where the supremum is achieved at the measure $P^\ast=\frac{e^C}{E_{P_0}[e^C]}P_0$. This can be verified by looking at $D_{KL}(P||P^\ast)$.

We say that a sequence of measures $\mu_0^\epsilon$ on $(\Omega, \mathcal F)$ satisfies the Laplace principle with rate $I$ if for all continuous and bounded real valued functions $f$ we have

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

Then using Bryc's lemma if we have exponential tightness we can say that $\mu^\epsilon$ satisfies a large deviations principle (LDP) with rate function $I$. Similarly, if $\mu^\epsilon$ satisfy a LDP with rate function $I$ and the measures satisfy a moment condition, then by Varadhan's lemma we can say that the Laplace principle holds. The relationship between large deviations and Laplace principle is clear to me.

However, applying the Donsker-Varadhan representation to the Laplace principle yields

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

It seems like there is some message here but I am unable to find it.

What is the relationship between $\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}$ and $\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}$?

For an example of a precise question, suppose that there is a sequence of optimizers of the measure supremum, call them $\mu^{\epsilon, \ast}$ and suppose that there is an optimizer $\omega^\ast$ in the other supremum. Then is it true that $\lim_{\epsilon\to 0} E_{\mu^{\epsilon,\ast}}[f]=f(\omega^\ast)$ and $\lim_{\epsilon\to 0} \epsilon D_{KL}(\mu^{\epsilon,\ast}||\mu_0^\epsilon)=I(\omega^\ast)$? This doesn't seem at all obvious why this should happen.

$\endgroup$
3
  • $\begingroup$ Yes, this is a feasible way to prove Large Deviation principle. See the book, Analysis and Approximation of Rare Events by Dupuis and Budhiraja, where this strategy is used to establish the Laplace principle in a wide variety of models. The idea is to interpret the former variational optimization as a stochastic control problem, while the latter problem is expressed as a deterministic control problem $\endgroup$
    – Miheer
    Commented Jun 3, 2023 at 13:43
  • 1
    $\begingroup$ @Miheer Thank you so much for the recommendation. This book is exactly what I've been looking for $\endgroup$
    – user479223
    Commented Jun 3, 2023 at 14:46
  • $\begingroup$ That's awesome. I am fascinated by this problem too! Thanks for formulating the question. $\endgroup$
    – Miheer
    Commented Jun 4, 2023 at 15:22

0

You must log in to answer this question.

Browse other questions tagged .