Sorry for such a newbie post and for asking two unrelated references in one shot.
First, I am interested in any proof of Pinsker's inequality.
Second, I wonder what is the best place to read about Pontryagin duality and harmonic analysis. To clarify, I took only standard functional analysis course.
Pinsker's inequality has many proofs. My favorites include Pollard's short but "magical" proof, https://www.cs.bgu.ac.il/~asml162/wiki.files/pollard-pinsker.pdf and surely the Proof from the Book is via Hoeffding's inequality+Fenchel-Lagrange duality, as in Theorem 2.16 Massart's book: http://www.cmap.polytechnique.fr/~merlet/articles/probas_massart_stf03.pdf
Folland's book ("A Course in Abstract Harmonic Analysis") is highly extensive, developing the machinery of spectral theory, Banach algebras, topological groups and the unitary representation theory of arbitrary LCH groups before arriving at the more specific theory of (locally compact Hausdorff) abelian groups. Later he also gives very general treatments of the theory of compact groups and of the construction of induced representations, and finally, he lays down the general results known for arbitrary LCSC (locally compact, second countable and Hausdorff) groups in harmonic analysis, mostly without proof. In my opinion this book is great but it is probably too heavy for you if you're just interested in the abelian theory. In that case Katznelson's "An Introduction to Harmonic Analysis" is a nice book which may be more suitable for you.
My favorite proof so far is the one from Theorem 4.5 of Yihong Wu's lecture notes, which uses the data processing inequality to reduce the problem to the binary case: if $S$ is a measurable subset of the domain, then we can consider the two random variable $\mathbf{1}_S$ under $P$ and $Q$, respectively, which has then distribution either $\mathrm{Bern}(P(S))$ or $\mathrm{Bern}(Q(S))$. First $$ \operatorname{TV}(\mathrm{Bern}(P(S)), \mathrm{Bern}(Q(S))) = |P(S)-Q(S)| \tag{1} $$ so we get the TV between $P$ and $Q$ by taking the supremum over all $S$. By the DPI, however, $$ \operatorname{KL}(\mathrm{Bern}(P(S))\ \|\ \mathrm{Bern}(Q(S))) \leq \operatorname{KL}(P\ \|\ Q) \tag{2} $$
This shows that it suffices to prove the binary case of Pinsker's, which is just a matter of proving a simple inequality: \begin{equation} 2(p-q)^2 \leq p\log\frac{p}{q} + (1-p)\log\frac{1-p}{1-q} \tag{3} \end{equation} The cases where either $p$ or $q$ is in $\{0,1\}$ are easily checked , so we can assume $p,q\in(0,1)$. To prove (3) in this case, we introduce the function $f\colon(0,1)\to\mathbb{R}$ defined by $f(x) = p\log x + (1-p)\log(1-x)$, and observe that the RHS of (3) is exactly $f(p)-f(q)$. We then can write $$ f(p)-f(q) = \int_{q}^p f'(x)dx = \int_{q}^p \frac{p-x}{x(1-x)}dx \geq 4\int_{q}^p (p-x)dx = 4\cdot\frac{1}{2}(p-q)^2 $$ which is (3) (in the middle, we used the fact that $x(1-x) \leq 1/4$ for $x\in(0,1)$).
I found Rudin's "Fourier Analysis on Groups" a good reference for Pontryagin duality. The level of functional analysis there is not too high.
proof of pinkser's inequality: http://www.clsp.jhu.edu/~sanjeev/520.674/notes/I-divergence-properties.pdf
I know that this question is quite old and there are some good suggestions here, but I just wanted to add the following references in case anyone else comes across this like I did:
In particular, they have good sections on locally compact groups (containing Pontryagin duality).
