The Kullback-Leibler Divergence (KLD) of two PMF's $P(x)$ and $Q(x)$ is $D(P||Q)=\sum_x P(x)\log(P(x)/Q(x))$, with the provisos that $0\cdot \log (0/p)=0$ and $p\cdot \log (p/0)=+\infty$ whenever $p>0$.

It is known that KLD is continuous at $(P,Q)$ if $Q$ is *strictly positive over all $x$'s*.  What can be said otherwise?

To be more specific, ssume we are given a sequence of PMF $\{(P_n,Q_n)\}_{n\geq 0}$ s.t. $(P_n,Q_n)\rightarrow (P,Q)$ in the simplex of PFM's (with the topology induced by, say, norm-1 distance).

Is it correct to deduce that

$\lim_{n\rightarrow \infty} D(P_n||Q_n) \geq D(P||Q)$ ?

This would follows if KLD is loewer-semicontinuous, right?


Many thanks.