Let $(X_n)_{n\in\mathbb{N}}$ be a sequence of i.i.d random variables with law $\mu$. The Sanov Theorem then states that the empirical measures $$ \mu^N =\frac{1}{N} \sum _{n=1}^N\delta _{X_n} $$ satisfy a large deviation principle at speed $N$ with good rate function $H(.\mid\mu)$, $H$ being the relative entropy.

I was wondering, what is known if we consider a sequence of independent random variables $(X_n)_{n\in\mathbb N}$ but with different laws $(\mu_n)_{n\in\mathbb N}$ ? For example, what if you take $X_n=Y_n+a_n$, where $(Y_n)_{n\in\mathbb n}$ is a sequence of i.i.d random variables with law $\mu$, and $(a_n)_{n\in\mathbb N}$ is a sequence of real numbers such that $$ \frac{1}{N} \sum _{n=1}^N\delta _{a_n}\rightarrow \nu \qquad \mbox{(weakly)} $$ for some probability measure $\nu$ as $N\rightarrow \infty$ ?

EDIT : (after the comment of Anthony Quas) Let's say we may assume the that the convergence rate for the $a_n$'s is as you want, for example at the rate $\exp(-N.)$. My interest is more about what would be the rate function.

  • 2
    $\begingroup$ You can't hope to get much of a rate unless you know something about how the $a_n$ converge to $\nu$. $\endgroup$ – Anthony Quas Mar 2 '12 at 14:44
  • $\begingroup$ @Anthony : You're right, see my Edit. $\endgroup$ – Adrien Hardy Mar 2 '12 at 15:07

In your case, I would suggest to try to show that the law of the empirical measures $$ \mu^N(X) = \frac{1}{N} \sum_{n=1}^N\delta_{X_n} $$ is very close to the law of $\mu^N(X')$ (quantify, based on the assumptions on $a_n$), where $X_n' = Y_n + A_n$, with $(A_n)_{n\ge 1}$ iid according to the law $\nu$, independent of $(Y_n)_{n\ge 1}$. Then use Sanov's theorem on $\mu^N(X')$.


You should use the G\"artner-Ellis theorem (GET), see (Dembo and Zeitouni section 2.3 or something). The fact that you still have independence means that the proposed log-moment generating function in GET will come out to being the asymptotic average of the log-moment generating function of the non indentical $X_n$.

  • $\begingroup$ This works whenever the $X_n$ only take a finite number of values, but can it be made rigorous in the general case? $\endgroup$ – Pascal Maillard Apr 18 '12 at 20:55
  • $\begingroup$ @Pascal Maillard, I don't think that Gartner-Ellis' theorem is restricted to finite alphabets, if that is what you are reffering to. $\endgroup$ – tipanverella Apr 19 '12 at 4:29
  • $\begingroup$ @tipanverella, sorry for the late answer, I wasn't notified of your comment, strangely. The Gartner-Ellis theorem is restricted to $\mathbb R^n$, as far as I know, and here we are considering the space of empirical measures on a measurable space $S$, which can be embedded into $\mathbb R^n$ only if $S$ is finite. Maybe one can work around it by discretizing the space, though... $\endgroup$ – Pascal Maillard Apr 28 '12 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.