Many wonderful answers here, but it is perhaps useful to note that Frenkel's setting has the feature that number fields are not "directly" entering the physics in the Langlands story (I dont know which particular talk the OP is referring, but I have a feeling what I say will be true). The setting for Langlands duality in physics is the study of four dimensional maximally supersymmetric Yang-Mills theory (usually called $\mathcal{N}=4$ SYM) with some gauge group G. This theory has a highly non-trivial (and still conjectured) duality which relates the strong coupling limit of this theory (with gauge group $G$) to weak coupling of $\mathcal{N}=4$ SYM with gauge group $G^\vee$. Now, these groups are compact Lie groups. The representation theory of the complex Lie Group enters the study of this gauge theory in various essential ways that I won't try to recall here. In the setup of Kapustin-Witten, this theory is the starting point and one studies this theory(+some decorations) under dimensional reduction (after a procedure called twisting) on a Riemann surface C down to two dimensions. Such a study leads one to the Geometric Column (Groups are over $\mathbb{C}$) of the Langlands story. The other settings to study the Langlands story can be : Curves over finite fields, or directly for Number fields. Frenkel usually writes these are different "columns" for the Langlands story.  Now, it happens that "suitable" statements made in the geometric setting can be taken over to other columns. But, the "suitable" caveat is important (and I am by no means an expert on when you can jump between columns). 

So, in the Langlands setting, it may be best to try think of the link between Number Theory and Physics as being a mediated one.

(Number Theory Questions addressed by Langlands)  ---- (Something) --- (Physics of a particular Supersymmetric Gauge Theory) 

This mediating is often done by Geometric Representation Theory (GRT). Here, the theory for infinite dimensional representations of the Complex Group plays an essential role. I'd be tempted to replace the "something" by GRT, but I will resist the temptation since I don't know the Number Theory setting well enough. One should also think of the above as a schematic and not as something that is a done deal. 

Back to the theme of the OP's question, to answer it, one may first ask if a Group over Number fields $G(\mathbb{Z})$ is directly entering a physics setup. This is often the case in examples. Many of the examples that people have noted fall in this category. Even the $\mathcal{N}=4$ theory has a $SL(2,\mathbb{Z})$ duality. But, in these scenarios, you (typically) don't have a lot of freedom to play with '$G$'. In the setting for geometric Langlands, you instead get a lot of freedom with '$G$' (any reductive group would do, as Langlands wants) but the connection to Number Theory itself is less direct (mediated as above). 

**Further Refs** : Here are a couple of [very][1] [good][2] posts on MO that explain things in more detail. Frenkel's book, "Love and Math" (cited by other answers as well) is also a fantastic source. 


  [1]: http://mathoverflow.net/questions/126474/new-geometric-methods-in-number-theory-and-automorphic-forms
  [2]: http://mathoverflow.net/questions/4180/consequences-of-geometric-langlands