5
$\begingroup$

Let $(X_n)_{n\in\mathbb{N}}$ be a sequence of i.i.d random variables with law $\mu$. The Sanov Theorem then states that the empirical measures $$ \mu^N =\frac{1}{N} \sum _{n=1}^N\delta _{X_n} $$ satisfy a large deviation principle at speed $N$ with good rate function $H(.\mid\mu)$, $H$ being the relative entropy.

I was wondering, what is known if we consider a sequence of independent random variables $(X_n)_{n\in\mathbb N}$ but with different laws $(\mu_n)_{n\in\mathbb N}$ ? For example, what if you take $X_n=Y_n+a_n$, where $(Y_n)_{n\in\mathbb n}$ is a sequence of i.i.d random variables with law $\mu$, and $(a_n)_{n\in\mathbb N}$ is a sequence of real numbers such that $$ \frac{1}{N} \sum _{n=1}^N\delta _{a_n}\rightarrow \nu \qquad \mbox{(weakly)} $$ for some probability measure $\nu$ as $N\rightarrow \infty$ ?

EDIT : (after the comment of Anthony Quas) Let's say we may assume the that the convergence rate for the $a_n$'s is as you want, for example at the rate $\exp(-N.)$. My interest is more about what would be the rate function.

$\endgroup$
2
  • 2
    $\begingroup$ You can't hope to get much of a rate unless you know something about how the $a_n$ converge to $\nu$. $\endgroup$ Mar 2, 2012 at 14:44
  • $\begingroup$ @Anthony : You're right, see my Edit. $\endgroup$ Mar 2, 2012 at 15:07

2 Answers 2

3
$\begingroup$

In your case, I would suggest to try to show that the law of the empirical measures $$ \mu^N(X) = \frac{1}{N} \sum_{n=1}^N\delta_{X_n} $$ is very close to the law of $\mu^N(X')$ (quantify, based on the assumptions on $a_n$), where $X_n' = Y_n + A_n$, with $(A_n)_{n\ge 1}$ iid according to the law $\nu$, independent of $(Y_n)_{n\ge 1}$. Then use Sanov's theorem on $\mu^N(X')$.

$\endgroup$
1
$\begingroup$

You should use the G\"artner-Ellis theorem (GET), see (Dembo and Zeitouni section 2.3 or something). The fact that you still have independence means that the proposed log-moment generating function in GET will come out to being the asymptotic average of the log-moment generating function of the non indentical $X_n$.

$\endgroup$
3
  • $\begingroup$ This works whenever the $X_n$ only take a finite number of values, but can it be made rigorous in the general case? $\endgroup$ Apr 18, 2012 at 20:55
  • $\begingroup$ @Pascal Maillard, I don't think that Gartner-Ellis' theorem is restricted to finite alphabets, if that is what you are reffering to. $\endgroup$ Apr 19, 2012 at 4:29
  • $\begingroup$ @tipanverella, sorry for the late answer, I wasn't notified of your comment, strangely. The Gartner-Ellis theorem is restricted to $\mathbb R^n$, as far as I know, and here we are considering the space of empirical measures on a measurable space $S$, which can be embedded into $\mathbb R^n$ only if $S$ is finite. Maybe one can work around it by discretizing the space, though... $\endgroup$ Apr 28, 2012 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.