MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $(X_n)_{n\in\mathbb{N}}$ be a sequence of i.i.d random variables with law $\mu$. The Sanov Theorem then states that the empirical measures $$ \mu^N =\frac{1}{N} \sum _{n=1}^N\delta _{X_n} $$ satisfy a large deviation principle at speed $N$ with good rate function $H(.\mid\mu)$, $H$ being the relative entropy.

I was wondering, what is known if we consider a sequence of independent random variables $(X_n)_{n\in\mathbb N}$ but with different laws $(\mu_n)_{n\in\mathbb N}$ ? For example, what if you take $X_n=Y_n+a_n$, where $(Y_n)_{n\in\mathbb n}$ is a sequence of i.i.d random variables with law $\mu$, and $(a_n)_{n\in\mathbb N}$ is a sequence of real numbers such that $$ \frac{1}{N} \sum _{n=1}^N\delta _{a_n}\rightarrow \nu \qquad \mbox{(weakly)} $$ for some probability measure $\nu$ as $N\rightarrow \infty$ ?

EDIT : (after the comment of Anthony Quas) Let's say we may assume the that the convergence rate for the $a_n$'s is as you want, for example at the rate $\exp(-N.)$. My interest is more about what would be the rate function.

share|cite|improve this question
You can't hope to get much of a rate unless you know something about how the $a_n$ converge to $\nu$. – Anthony Quas Mar 2 '12 at 14:44
@Anthony : You're right, see my Edit. – Adrien Hardy Mar 2 '12 at 15:07

In your case, I would suggest to try to show that the law of the empirical measures $$ \mu^N(X) = \frac{1}{N} \sum_{n=1}^N\delta_{X_n} $$ is very close to the law of $\mu^N(X')$ (quantify, based on the assumptions on $a_n$), where $X_n' = Y_n + A_n$, with $(A_n)_{n\ge 1}$ iid according to the law $\nu$, independent of $(Y_n)_{n\ge 1}$. Then use Sanov's theorem on $\mu^N(X')$.

share|cite|improve this answer

You should use the G\"artner-Ellis theorem (GET), see (Dembo and Zeitouni section 2.3 or something). The fact that you still have independence means that the proposed log-moment generating function in GET will come out to being the asymptotic average of the log-moment generating function of the non indentical $X_n$.

share|cite|improve this answer
This works whenever the $X_n$ only take a finite number of values, but can it be made rigorous in the general case? – Pascal Maillard Apr 18 '12 at 20:55
@Pascal Maillard, I don't think that Gartner-Ellis' theorem is restricted to finite alphabets, if that is what you are reffering to. – tipanverella Apr 19 '12 at 4:29
@tipanverella, sorry for the late answer, I wasn't notified of your comment, strangely. The Gartner-Ellis theorem is restricted to $\mathbb R^n$, as far as I know, and here we are considering the space of empirical measures on a measurable space $S$, which can be embedded into $\mathbb R^n$ only if $S$ is finite. Maybe one can work around it by discretizing the space, though... – Pascal Maillard Apr 28 '12 at 20:18

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.