Many wonderful answers here, but it is perhaps useful to note that Frenkel's setting has the feature that number fields are not "directly" entering the physics in the Langlands story (I dont know which particular talk the OP is referring, but I have a feeling what I say will be true). The setting for Langlands duality in physics is the study of four dimensional maximally supersymmetric Yang-Mills theory (usually called $\mathcal{N}=4$ SYM) with some gauge group G. This theory has a highly non-trivial (and still conjectured) *S*-duality which relates the the theory with gauge group $G$ and coupling $\tau$ to  $\mathcal{N}=4$ SYM with gauge group $G^\vee$ and coupling $-1/\tau n_r$, where $n_r$ is the ratio of the length of the long root to that of the short root in the root system of $G$. Now, these groups are compact Lie groups. The representation theory of the complex Lie Group enters the study of this gauge theory in various essential ways that I won't try to recall here. In the setup of Kapustin-Witten, this theory is the starting point and one studies this theory(+some decorations) under dimensional reduction (after a procedure called twisting) on a Riemann surface C down to two dimensions. Such a study leads one to the Geometric Column (Groups are over $\mathbb{C}$) of the Langlands story. The other settings to study the Langlands story can be : Curves over finite fields, or directly for Number fields. Frenkel usually writes these are different "columns" for the Langlands story.  Now, it happens that "suitable" statements made in the geometric setting can be taken over to other columns. But, the "suitable" caveat is important (and I am by no means an expert on when you can jump between columns). 

So, in the Langlands setting, it may be best to try think of the link between Number Theory and Physics as being a mediated one.

(Number Theory Questions addressed by Langlands)  ---- (Something) --- (Physics of a particular Supersymmetric Gauge Theory) 

This mediating is often done by Geometric Representation Theory (GRT). Here, the theory for infinite dimensional representations of the Complex Group plays an essential role. I'd be tempted to replace the "something" by GRT, but I will resist the temptation since I don't know the Number Theory setting well enough. One should also think of the above as a schematic and not as something that is a done deal. 

Back to the theme of the OP's question, to answer it, one may first ask if a Group over Number fields $G(\mathbb{Z})$ is directly entering a physics setup. This is often the case in examples. Many of the examples that people have noted fall in this category. Even the $\mathcal{N}=4$ theory has a $SL(2,\mathbb{Z})$ duality. The S-duality transformation recalled above is a particular element in this full $SL(2,\mathbb{Z})$ duality. Now, the $SL_2$ appearing in the duality group here is very interesting, but this is _not_ the group identified (in the KW setup) with the reductive group needed for the Langlands story. 

  This example illustrates a useful point that in cases where a $G(\mathbb{Z})$ is entering the physics directly, you typically don't have a lot of freedom to play with what the '$G$' can be. However, in setting for the Gauge Theory approach to Geometric Langlands, the reductive Group needed for Langlands is identified with a Gauge Group of a 4d QFT. Now, you get a lot of freedom for what '$G$' can be. But the connection to Number Theory itself is less direct (mediated as above). 

**Further Refs** : Here are a couple of [very][1] [good][2] posts on MO that explain things in more detail. Frenkel's book, "Love and Math" (cited by other answers as well) is also a fantastic source. 


  [1]: https://mathoverflow.net/questions/126474/new-geometric-methods-in-number-theory-and-automorphic-forms
  [2]: https://mathoverflow.net/questions/4180/consequences-of-geometric-langlands