# A generalization of the Sanov Theorem

Let $(X_n)_{n\in\mathbb{N}}$ be a sequence of i.i.d random variables with law $\mu$. The Sanov Theorem then states that the empirical measures $$\mu^N =\frac{1}{N} \sum _{n=1}^N\delta _{X_n}$$ satisfy a large deviation principle at speed $N$ with good rate function $H(.\mid\mu)$, $H$ being the relative entropy.

I was wondering, what is known if we consider a sequence of independent random variables $(X_n)_{n\in\mathbb N}$ but with different laws $(\mu_n)_{n\in\mathbb N}$ ? For example, what if you take $X_n=Y_n+a_n$, where $(Y_n)_{n\in\mathbb n}$ is a sequence of i.i.d random variables with law $\mu$, and $(a_n)_{n\in\mathbb N}$ is a sequence of real numbers such that $$\frac{1}{N} \sum _{n=1}^N\delta _{a_n}\rightarrow \nu \qquad \mbox{(weakly)}$$ for some probability measure $\nu$ as $N\rightarrow \infty$ ?

EDIT : (after the comment of Anthony Quas) Let's say we may assume the that the convergence rate for the $a_n$'s is as you want, for example at the rate $\exp(-N.)$. My interest is more about what would be the rate function.

-
You can't hope to get much of a rate unless you know something about how the $a_n$ converge to $\nu$. –  Anthony Quas Mar 2 '12 at 14:44
@Anthony : You're right, see my Edit. –  Adrien Hardy Mar 2 '12 at 15:07

In your case, I would suggest to try to show that the law of the empirical measures $$\mu^N(X) = \frac{1}{N} \sum_{n=1}^N\delta_{X_n}$$ is very close to the law of $\mu^N(X')$ (quantify, based on the assumptions on $a_n$), where $X_n' = Y_n + A_n$, with $(A_n)_{n\ge 1}$ iid according to the law $\nu$, independent of $(Y_n)_{n\ge 1}$. Then use Sanov's theorem on $\mu^N(X')$.
You should use the G\"artner-Ellis theorem (GET), see (Dembo and Zeitouni section 2.3 or something). The fact that you still have independence means that the proposed log-moment generating function in GET will come out to being the asymptotic average of the log-moment generating function of the non indentical $X_n$.
This works whenever the $X_n$ only take a finite number of values, but can it be made rigorous in the general case? –  Pascal Maillard Apr 18 '12 at 20:55
@tipanverella, sorry for the late answer, I wasn't notified of your comment, strangely. The Gartner-Ellis theorem is restricted to $\mathbb R^n$, as far as I know, and here we are considering the space of empirical measures on a measurable space $S$, which can be embedded into $\mathbb R^n$ only if $S$ is finite. Maybe one can work around it by discretizing the space, though... –  Pascal Maillard Apr 28 '12 at 20:18