CLT for random variables with positive support (e.g. exponential)

Question

I have a bunch of iid $\{X_i\}$ with $X_i \sim \exp(\lambda)$ - let's say $\lambda = 1$. Now, classic version of CLT tells me: \begin{equation} \sqrt{n}\left(1-\bar{X}_n\right) \rightarrow \mathcal{N}\left(0,\frac{1}{\lambda^2}\right) \end{equation} in distribution. But doesn't the convergence to a standard normal implies a probability $> 0$ of negative draws of the sample mean? - Which cannot be drawn as the exponential distribution has a positive support.

I am curious. And for a practical reason as well, because as in my real world example I want to apply Baysian estimate for the mean of a random variable with non-negative support. My assumption is that I have iid $\{X_i\}$ and a known variance $\sigma^2$. I want to estimate the mean of the RV. Now, the Likelihood is normal and the informative prior as well.

But if feels weird to apply a prior with negative support on to estimate the mean of a random variable with positive support. And I guess the situation is not changed by applying a Normal-inverse-gamma distribution as prior.

Answer 1

You need to be careful with the order of quantifiers in understanding what (this version of) CLT is claiming. In particular, the order of $x$ (the point where you evaluate your CDF) and $n$ (sample size).

Convergence in distribution means that if you pick a value $x$ (yes, it can be negative), and consider the value of $F_n(x) = P(\sqrt{n}(\overline{X}_n - 1) \le x)$ at that point, then that value tends to $\Phi(x)$ as $n \to \infty$. Yes, that limit is positive (for all $x$).

This is indeed what happens. Let's be super concrete and consider $x=-3$. Can you have $\sqrt{n}(\overline{X}_n-1) \le -3$? Yes, consider for example $n=10$. Your sample mean $\overline{X}_n$ could be $0.001$, and then your $\sqrt{n}(\overline{X}_n-1)$ would be $-3.159$. There is a whole range of possible sample means that map to below $-3$, so indeed that event has a positive probability. Not a big one, but positive. It tends to $\Phi(-3) \approx 0.0013$ as $n \to \infty$. (Keeping $x$ fixed at $-3$.)

You can repeat the same experiment with some other value of $x$, say $-10$. Then you'll find that for $n$ large enough, the deviation of your sample mean, scaled by multiplying by $\sqrt{n}$, could indeed be that much negative.