# Two reference requests: Pinsker's inequality and Pontryagin duality

Sorry for such a newbie post and for asking two unrelated references in one shot.

First, I am interested in any proof of Pinsker's inequality.

Second, I wonder what is the best place to read about Pontryagin duality and harmonic analysis. To clarify, I took only standard functional analysis course.

• Why not split into two questions? For the second, I recommend "Fourier Analysis on Number Fields." Commented Oct 18, 2010 at 16:57
• What is the point in splitting? Commented Oct 18, 2010 at 17:02
• @ilyaraz Factorizing polynomials. Commented Oct 18, 2010 at 17:47
• You might also enjoy the following paper on Generalized Pinsker Inequalities: mark.reid.name/files/pubs/colt09.pdf Commented Oct 18, 2010 at 20:01
• Cam: ilyaraz may be more of an analyst than a number theorist, so the book you recommended might not be to ilyaraz's taste. Perhaps Folland's "A Course in Abstract Harmonic Analysis" as an alternative? Commented Oct 19, 2010 at 5:23

Pinsker's inequality has many proofs. My favorites include Pollard's short but "magical" proof, https://www.cs.bgu.ac.il/~asml162/wiki.files/pollard-pinsker.pdf and surely the Proof from the Book is via Hoeffding's inequality+Fenchel-Lagrange duality, as in Theorem 2.16 Massart's book: http://www.cmap.polytechnique.fr/~merlet/articles/probas_massart_stf03.pdf

Folland's book ("A Course in Abstract Harmonic Analysis") is highly extensive, developing the machinery of spectral theory, Banach algebras, topological groups and the unitary representation theory of arbitrary LCH groups before arriving at the more specific theory of (locally compact Hausdorff) abelian groups. Later he also gives very general treatments of the theory of compact groups and of the construction of induced representations, and finally, he lays down the general results known for arbitrary LCSC (locally compact, second countable and Hausdorff) groups in harmonic analysis, mostly without proof. In my opinion this book is great but it is probably too heavy for you if you're just interested in the abelian theory. In that case Katznelson's "An Introduction to Harmonic Analysis" is a nice book which may be more suitable for you.

I found Rudin's "Fourier Analysis on Groups" a good reference for Pontryagin duality. The level of functional analysis there is not too high.

proof of pinkser's inequality: http://www.clsp.jhu.edu/~sanjeev/520.674/notes/I-divergence-properties.pdf

• The link doesn't work for me. Commented Oct 26, 2018 at 8:29

I know that this question is quite old and there are some good suggestions here, but I just wanted to add the following references in case anyone else comes across this like I did:

• "An Introduction to Abstract Harmonic Analysis" by Loomis
• "Classical Harmonic Analysis and Locally Compact Groups" by Reiter

In particular, they have good sections on locally compact groups (containing Pontryagin duality).

My favorite proof so far is the one from Theorem 4.5 of Yihong Wu's lecture notes, which uses the data processing inequality to reduce the problem to the binary case: if $$S$$ is a measurable subset of the domain, then we can consider the two random variable $$\mathbf{1}_S$$ under $$P$$ and $$Q$$, respectively, which has then distribution either $$\mathrm{Bern}(P(S))$$ or $$\mathrm{Bern}(Q(S))$$. First $$\operatorname{TV}(\mathrm{Bern}(P(S)), \mathrm{Bern}(Q(S))) = |P(S)-Q(S)| \tag{1}$$ so we get the TV between $$P$$ and $$Q$$ by taking the supremum over all $$S$$. By the DPI, however, $$\operatorname{KL}(\mathrm{Bern}(P(S))\ \|\ \mathrm{Bern}(Q(S))) \leq \operatorname{KL}(P\ \|\ Q) \tag{2}$$

This shows that it suffices to prove the binary case of Pinsker's, which is just a matter of proving a simple inequality: $$$$2(p-q)^2 \leq p\log\frac{p}{q} + (1-p)\log\frac{1-p}{1-q} \tag{3}$$$$ The cases where either $$p$$ or $$q$$ is in $$\{0,1\}$$ are easily checked , so we can assume $$p,q\in(0,1)$$. To prove (3) in this case, we introduce the function $$f\colon(0,1)\to\mathbb{R}$$ defined by $$f(x) = p\log x + (1-p)\log(1-x)$$, and observe that the RHS of (3) is exactly $$f(p)-f(q)$$. We then can write $$f(p)-f(q) = \int_{q}^p f'(x)dx = \int_{q}^p \frac{p-x}{x(1-x)}dx \geq 4\int_{q}^p (p-x)dx = 4\cdot\frac{1}{2}(p-q)^2$$ which is (3) (in the middle, we used the fact that $$x(1-x) \leq 1/4$$ for $$x\in(0,1)$$).

• $x(1-x) \leq 4$ for all $x$. In short, I belive the inequality in (3) holds for all $p,q$, not just $p,q \in [0,1]$. Commented Nov 30, 2023 at 17:21
• @travelingbones $p,q$ are Bernoulli parameters in the proof, so there's not need to consider values outside of $[0,1]$. And the RHS of (3) is not defined for, say, $p>1$ and $q \in(0,1)$ (logarithm of a negative number) Commented Nov 30, 2023 at 19:58
• Concur on equation (3). Commented Nov 30, 2023 at 21:05
• Here is a currently working link to the full proof (as the one above if no longer working): home.ttic.edu/~madhurt/courses/infotheory2014/l5.pdf Commented Nov 30, 2023 at 21:06