Skip to main content
Notice removed Draw attention by CommunityBot
Bounty Ended with no winning answer by CommunityBot
Notice added Draw attention by user479223
Bounty Started worth 100 reputation by user479223
Notice removed Draw attention by CommunityBot
Bounty Ended with no winning answer by CommunityBot
Notice added Draw attention by user479223
Bounty Started worth 50 reputation by user479223
added 462 characters in body
Source Link
user479223
  • 1.9k
  • 2
  • 9
  • 30

Given a nice real valued functional $C$ on some probability space $(\Omega, \mathcal F, P_0)$ we have the following Donsker-Varadhan variational representation

$$\log E_{P_0}\left[e^C\right]=\sup_{P\sim P_0}\{E_P[C]-D_{KL}(P||P_0)\},$$

where the supremum is achieved at the measure $P^\ast=\frac{e^C}{E_{P_0}[e^C]}P_0$. This can be verified by looking at $D_{KL}(P||P^\ast)$.

We say that a sequence of measures $\mu_0^\epsilon$ on $(\Omega, \mathcal F)$ satisfies the Laplace principle with rate $I$ if for all continuous and bounded real valued functions $f$ we have

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

Then using Bryc's lemma if we have exponential tightness we can say that $\mu^\epsilon$ satisfies a large deviations principle (LDP) with rate function $I$. Similarly, if $\mu^\epsilon$ satisfy a LDP with rate function $I$ and the measures satisfy a moment condition, then by Varadhan's lemma we can say that the Laplace principle holds. The relationship between large deviations and Laplace principle is clear to me.

However, applying the Donsker-Varadhan representation to the Laplace principle yields

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

It seems like there is some message here but I am unable to find it.

What is the relationship between $\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}$ and $\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}$?

For an example of a precise question, suppose that there is a sequence of optimizers of the measure supremum, call them $\mu^{\epsilon, \ast}$ and suppose that there is an optimizer $\omega^\ast$ in the other supremum. Then is it true that $\lim_{\epsilon\to 0} E_{\mu^{\epsilon,\ast}}[f]=f(\omega^\ast)$ and $\lim_{\epsilon\to 0} \epsilon D_{KL}(\mu^{\epsilon,\ast}||\mu_0^\epsilon)=I(\omega^\ast)$? This doesn't seem at all obvious why this should happen.

Given a nice real valued functional $C$ on some probability space $(\Omega, \mathcal F, P_0)$ we have the following Donsker-Varadhan variational representation

$$\log E_{P_0}\left[e^C\right]=\sup_{P\sim P_0}\{E_P[C]-D_{KL}(P||P_0)\},$$

where the supremum is achieved at the measure $P^\ast=\frac{e^C}{E_{P_0}[e^C]}P_0$. This can be verified by looking at $D_{KL}(P||P^\ast)$.

We say that a sequence of measures $\mu_0^\epsilon$ on $(\Omega, \mathcal F)$ satisfies the Laplace principle with rate $I$ if for all continuous and bounded real valued functions $f$ we have

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

Then using Bryc's lemma if we have exponential tightness we can say that $\mu^\epsilon$ satisfies a large deviations principle (LDP) with rate function $I$. Similarly, if $\mu^\epsilon$ satisfy a LDP with rate function $I$ and the measures satisfy a moment condition, then by Varadhan's lemma we can say that the Laplace principle holds. The relationship between large deviations and Laplace principle is clear to me.

However, applying the Donsker-Varadhan representation to the Laplace principle yields

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

It seems like there is some message here but I am unable to find it.

What is the relationship between $\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}$ and $\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}$?

Given a nice real valued functional $C$ on some probability space $(\Omega, \mathcal F, P_0)$ we have the following Donsker-Varadhan variational representation

$$\log E_{P_0}\left[e^C\right]=\sup_{P\sim P_0}\{E_P[C]-D_{KL}(P||P_0)\},$$

where the supremum is achieved at the measure $P^\ast=\frac{e^C}{E_{P_0}[e^C]}P_0$. This can be verified by looking at $D_{KL}(P||P^\ast)$.

We say that a sequence of measures $\mu_0^\epsilon$ on $(\Omega, \mathcal F)$ satisfies the Laplace principle with rate $I$ if for all continuous and bounded real valued functions $f$ we have

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

Then using Bryc's lemma if we have exponential tightness we can say that $\mu^\epsilon$ satisfies a large deviations principle (LDP) with rate function $I$. Similarly, if $\mu^\epsilon$ satisfy a LDP with rate function $I$ and the measures satisfy a moment condition, then by Varadhan's lemma we can say that the Laplace principle holds. The relationship between large deviations and Laplace principle is clear to me.

However, applying the Donsker-Varadhan representation to the Laplace principle yields

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

It seems like there is some message here but I am unable to find it.

What is the relationship between $\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}$ and $\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}$?

For an example of a precise question, suppose that there is a sequence of optimizers of the measure supremum, call them $\mu^{\epsilon, \ast}$ and suppose that there is an optimizer $\omega^\ast$ in the other supremum. Then is it true that $\lim_{\epsilon\to 0} E_{\mu^{\epsilon,\ast}}[f]=f(\omega^\ast)$ and $\lim_{\epsilon\to 0} \epsilon D_{KL}(\mu^{\epsilon,\ast}||\mu_0^\epsilon)=I(\omega^\ast)$? This doesn't seem at all obvious why this should happen.

Source Link
user479223
  • 1.9k
  • 2
  • 9
  • 30

What exactly is the relationship between Donsker-Varadhan variational formula and the Laplace principle?

Given a nice real valued functional $C$ on some probability space $(\Omega, \mathcal F, P_0)$ we have the following Donsker-Varadhan variational representation

$$\log E_{P_0}\left[e^C\right]=\sup_{P\sim P_0}\{E_P[C]-D_{KL}(P||P_0)\},$$

where the supremum is achieved at the measure $P^\ast=\frac{e^C}{E_{P_0}[e^C]}P_0$. This can be verified by looking at $D_{KL}(P||P^\ast)$.

We say that a sequence of measures $\mu_0^\epsilon$ on $(\Omega, \mathcal F)$ satisfies the Laplace principle with rate $I$ if for all continuous and bounded real valued functions $f$ we have

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

Then using Bryc's lemma if we have exponential tightness we can say that $\mu^\epsilon$ satisfies a large deviations principle (LDP) with rate function $I$. Similarly, if $\mu^\epsilon$ satisfy a LDP with rate function $I$ and the measures satisfy a moment condition, then by Varadhan's lemma we can say that the Laplace principle holds. The relationship between large deviations and Laplace principle is clear to me.

However, applying the Donsker-Varadhan representation to the Laplace principle yields

$$\lim_{\epsilon\to 0}\epsilon \log E_{\mu_0^\epsilon}\left[e^{f/\epsilon}\right]=\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}=\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}.$$

It seems like there is some message here but I am unable to find it.

What is the relationship between $\lim_{\epsilon\to 0} \sup_{\mu^\epsilon\sim \mu_0^\epsilon}\{E_{\mu^\epsilon}[f]-\epsilon D_{KL}(\mu^\epsilon||\mu_0^\epsilon)\}$ and $\sup_{\omega\in\Omega}\{f(\omega)-I(\omega)\}$?