Last Inference in proof of conditional limit theorem

Question

I read about the Conditional Limit Theorem from the book "Elements of Information Theory" by Thomas M. Cover and Joy A. Thomas, second edition, page 371. I can't understand the last inference in the proof of the theorem.

The theorem is formulated as follows: Let $E$ be a closed convex subset of probability distributions and let $Q$ be a distribution not in $E$. Let $X_1,X_2,\dots,X_n$ be discrete random variables drawn i.i.d ~ $Q$. Let $P^*$ achieve $\min_{P \in E}D(P||Q)$. Then: $$\mathrm{Pr}\left(X_1 = a \big\vert P_{X^n} \in E\right) \rightarrow P^*(a)$$ in probability, as $n \rightarrow \infty$.

Note: In the formulation above, $P_{X^n}$ means the "type" of the sequence $X^n = (X_1,X_2,\dots,X_n)$.

The end of the proof contains the following statement:

Thus, $\mathrm{Pr}\left(\left|P_{X^n}(a) - P^*(a)\right| \geq \epsilon \big\vert P_{X^n} \in E\right) \rightarrow 0$ as $n \rightarrow \infty$ (Here |.| is used to denote $L_1$ distance). Alternatively, this can be written as $\mathrm{Pr}\left(X_1 = a \big\vert P_{X^n} \in E\right) \rightarrow P^*(a)$ in probability, $a \in \chi$.

I don't understand how to conclude this last inference (in bold).

Answer 1

First of all, |.| does not denote the $L_1$ distance, but the absolute value. Second, the statement follows from the definition of convergence in probability, see for example page 58 in the same book.

Answer 2

I got stuck here too today, so for the reference of future readers, we use the following fact here:

Suppose for $1 \leq i \leq n$, $X_i \sim Q$ iid, then for every $k$ we have $\Pr [X_k =a | P_{X^n} = p] = p(a)$.

Which basically says that given the empirical probability distribution, all the letters are distributed according to the empirical probability distribution. This can be proven by simply expanding the LHS exactly using combinatorics.

I was able to go from this fact to the required statement using 3-4 steps requiring basic limit calculus.