On an asymptotic formula of Keating and Snaith involving the Riemann zeta function Keating and Snaith have a famous conjecture on the asymptotics of the
integral $\int_0^T |\zeta(\frac 12+it)|^{2k}\, dt$, where $\zeta$
denotes the Riemann zeta function. See page 510 of the book review by
Brian Conrey of H. Iwaniec, Lectures on the Riemann zeta function in
Bull. Amer. Math. Soc. 53 (2016), 507--512
(http://www.ams.org/journals/bull/2016-53-03/S0273-0979-2016-01525-4/S0273-0979-2016-01525-4.pdf). This
conjecture involves a certain number $g_k$. This number is equal to
the number of standard Young tableaux whose shape is a $k\times k$
rectangle. Equivalently, this is the degree of the irreducible
representation of the symmetric group $S_{k^2}$ corresponding to this
shape. Is there any (conjectural) explanation for this connection
between $\zeta(s)$ and the representation theory of $S_{k^2}$? Does the
degree of other irreps of $S_n$ (for suitable $n$) also have
connections with the asymptotics of $\zeta(s)$?
 A: There is a connection! (Though see the edit below.) Keating and Snaith make their conjecture by modeling the distribution of $\zeta(s)$ by the distribution of the characteristic polynomial of a random unitary matrix, distributed according to Haar measure. This is connected to symmetric function theory by Schur-Weyl duality. 
To my knowledge the first time this connection was used to compute moments (of a characteristic polynomial) was in a beautiful paper of Bump and Gamburd. See in particular Prop 4 (and perhaps especially the quick proof) and Corollary 1. 
As to other connections, slightly different averages of the zeta function, such as $\frac{\zeta(\alpha+s)\zeta(\beta+\overline{s})}{\zeta(\gamma+s)\zeta(\delta+\overline{s})}$ are also be conjectured to be related to analogous averages in random matrix theory, the computation of which can be reduced to symmetric function theory in a similar way. This too is done in later sections of the Bump and Gamburd paper.
There is also a very nice paper of Dehaye that you might find interesting expressing lower order terms in conjectures for the moments of $\zeta(s)$ in terms of counts of taubleaux.
Edit: it occurs to me that I may be leaving something unsaid. In particular you ask specifically about $S_{k^2}$. The paper of Bump and Gamburd shows in a pleasant and transparent enough way that
$$
\int_{U(n)} |\det(1-g)|^{2k}\, dg = s_{\langle n^k \rangle}(1^{2k}).
$$
To see what this should say about the zeta function we need to know that $s_{\langle n^k \rangle}(1^{2k}) \sim g_k n^{k^2}/(k^2)!$, where $g_k$ let's say is defined as the count of tableaux you give. While it is plausible that once symmetric function theory is involved more symmetric function theory will come into play, and while there are various ways to prove this, I don't know of any proof that makes this asymptotic relationship entirely transparent. I'd be happy to learn of one.
It may also be that you're wondering whether there is a way to get from number theory to the symmetric function theory without passing through the middleman of random matrix theory. I think this is a very interesting question but I don't have anything conclusive to say about it. One remark that can be made is that the conjectures of Conrey, Farmer, Keating, Rubinstein, and Snaith that Dehaye uses in his paper came out of a recipe that keeps track of various symmetries in products of the approximate functional equation for the zeta function (identity (36) in Dehaye's paper).
