This is a long comment rather than an answer. Also, I will be very sloppy about the details because I don't have a strong knowledge about them.
I think the fact that Pontryagin duality works with maps $\mathrm{Hom}(\,\_\,,\mathbb{T})$ where $\mathbb{T}=U(1)$ has to do with the following facts:
- Cartier duality, which is a very natural/reasonable thing to do, works with the multiplicative group, wich is $\mathbb{C}^\times$ over the complex numbers.
- $\mathbb{T}\subset\mathbb{C}^\times$ is the compact real form of the reductive group $\mathbb{C}^\times$, and the representation theory of a complex reductive algebraic group is "the same" as that of its compact real form.
[Important edit: as the comment by Yemon Choi reminded me, in what follows the algebraic group $G$ has to be diagonalizable, or at least a version of Cartier duality that I'm aware of is stated for such group schemes in Waterhouse Introduction to affine group schemes. This is very restrictive!]
So what's Cartier duality? Working over $\mathbb{C}$, there is an involution on the category $\mathbf{CHopf}$ of commutative Hopf algebras
$$\mathbf{CHopf}\to\mathbf{CHopf}$$
$$A:=(V,m,\Delta)\mapsto A^\vee:=(V^\vee,\Delta^\vee,m^\vee)$$
sending a Hopf algebra $A$ with underlying $\mathbb{C}$-vector space $V$, multiplication $m$ and comultiplication $\Delta$ (and other data that I'm suppressing from the notation) to the Hopf algebra structure $A^\vee$ on the dual vector space $V^\vee=\mathrm{Hom}(V,\mathbb{C})$ with multiplication given by $\Delta^\vee$ and comultiplication by $m^\vee$. The category $\mathbf{aaGrp}$ of affine complex algebraic groups is opposite to the category $\mathbf{CHopf}$ via $A=(V,m,\Delta)\mapsto G:=\mathrm{Spec}(V,m)$ with group operation given by the morphism $\mathrm{Spec}(\Delta):G\times G\to G$. Vice versa, an algebraic group $(G,\mu)$ goes to $V=\mathcal{O}(G)$, the algebra of global functions on $G$, with its natural algebra structure, and comultiplication given by $\mu^{\sharp}:\mathcal{O}(G)\to\mathcal{O}(G\times G)=\mathcal{O}(G)\otimes\mathcal{O}(G)$.
Under this antiequivalence $\mathbf{CHopf}\to\mathbf{aaGrp}$, the involution $A\to A^\vee$ corresponds to $G\mapsto \mathbf{aaGrp}(G,\mathbb{C}^\times)$.
This is to say that "Pontryagin-dualizing" with respect to $\mathbb{T}$ somehow comes from the usual duality of $\mathbb{C}$-vector spaces.