What is the normalizer of ${\mathrm{SL}}_2({\Bbb Z})$ in ${\mathrm{GL}}_2({\Bbb R})$? Namely ${\mathrm{N}}_{{\mathrm{GL}}_2({\Bbb R})}({\mathrm{SL}}_2({\Bbb Z}))$?

3$\begingroup$ Please edit the header so it agrees with the actual question asked. This has caused confusion. $\endgroup$– Jim HumphreysCommented Feb 21, 2014 at 13:38

1$\begingroup$ mathoverflow.net/questions/60654/… $\endgroup$– Marc PalmCommented Feb 21, 2014 at 15:58
4 Answers
The normalizer in $SL_2(\Bbb R)$ is indeed $SL_2(\Bbb Z)$.
[See the comments by Yves Cornulier for the normalizer in $GL_2(\Bbb R)$.]
If $\pmatrix{a&b\\c&d}\in SL_2(\Bbb R)$ normalizes $SL_2(\Bbb Z)$, then it also normalizes the $\Bbb Z$span of $SL_2(\Bbb Z)$, which is the full ring of integer two by two matrices. Thus one finds that $a^2$, $ab$, $ac$, $ad$, $b^2$, $bc$, $bd$, $c^2$, $cd$, $d^2$ are all integers. As explained by Igor this means that there are integers $x$, $y$, $z$, $w$, $t$, $t\geq0$ so that $a=x\sqrt t$, $b=y \sqrt t$, $c=z\sqrt t$, $d=w\sqrt t$. Now $adbc=1$ implies that $\pmatrix{a&b\\c&d}$ lies in $SL_2(\Bbb Z)$.

$\begingroup$ The normalizer obviously includes scalar matrices and $GL_2(\mathbb{Z})$. $\endgroup$ Commented Feb 21, 2014 at 10:17

$\begingroup$ He is calculating the normalizer in ${\rm SL}(2,{\mathbb R})$. $\endgroup$ Commented Feb 21, 2014 at 10:42

$\begingroup$ This provides the normalizer in $SL_2(\mathbf{R})$. Still, this immediately entails that the normalizer in $GL_2(\mathbf{R})$ is reduced to $\mathbf{R}^*GL_2(\mathbf{Z})$. $\endgroup$– YCorCommented Feb 21, 2014 at 10:43

1$\begingroup$ Sorry, I was answering the question in the title, which is different from the question in the body. $\endgroup$ Commented Feb 21, 2014 at 10:45

2$\begingroup$ Since the title has been changed, here is how the full answer follows: if a $2\times 2$ matrix over $\mathbf{R}$ normalizes $SL_2(\mathbf{Z})$, we can multiply it by a scalar matrix to get a matrix with determinant $\pm 1$, and then by some matrix in $GL_2(\mathbf{Z})$ to get a matrix in $SL_2(\mathbf{R})$. By Wilberd's answer, the latter is in $SL_2(\mathbf{Z})$, which means the original normalizing matrix had the form $\lambda A$ with $\lambda\in\mathbf{R}^*$ and $A\in GL_2(\mathbf{Z})$. $\endgroup$– YCorCommented Feb 22, 2014 at 19:56
The normalizer of $\Gamma:=\mathrm{PSL}_2(\mathbb{\mathbb{Z}})$ in $\mathrm{PGL}_2(\mathbb{\mathbb{R}})$ is $\Gamma':=\mathrm{PGL}_2(\mathbb{\mathbb{Z}})$. Hence, the answer to the original question is something like $\mathrm{GL}_2(\mathbb{\mathbb{Z}})\times\{\text{scalar matrices}\}$.
For the former assertion, observe that the only outer automorphism of $\Gamma$ is the conjugation by (any) element of $\Gamma'\setminus\Gamma$. Hence, modulo $\Gamma'$, it suffices to list matrices commuting with the two standard generators (e.g., $L$ and $R$) of $\Gamma$. A few lines in Maple show that the only such matrix is $I$.
The statement on the outer automorphisms is well known; here is an outline of a simple proof I can think of right now (at least, for trace preserving automorphisms, which is our case). An element $A\ne I$ of $\Gamma$ is parabolic iff its trace equals $2$ iff it has a single eigenvector iff it has an invariant vector. All statements below are easy exercises.
Fact 1: All parabolic elements with a given eigenvector form a cyclic group, which is the stabilizer of this vector.
Hint: assume the vector $[1,0]$.
Thus, we can speak about minimal parabolic elements, i.e., the generators of the corresponding cyclic groups.
Fact 2: Two minimal parabolic elements generate $\Gamma$ if and only if their corresponding invariant vectors $v_1$, $v_2$ form a basis for $\mathbb{Z}^2$.
Hint for "only if": parabolic elements being transvections, the subgroup spanned by $v_1$, $v_2$ would remain invariant. On the other hand, $\Gamma$ is transitive.
Now, the automorphism takes the standard parabolic generators $L,R$ to a pair of parabolic elements whose eigenvectors $v_1$, $v_2$ form a basis. On the other hand, $\Gamma$ is transitive on the set of positive bases. Hence, the new generators can be conjugated back to $L^{\pm1}$, $R^{\pm1}$. The rest is immediate.

1$\begingroup$ How do you justify the fact that the only outer automorphism of ${\rm PSL}(2,\mathbb{Z})$ is that induced by the action of ${\rm PGL}$? $\endgroup$ Commented Feb 21, 2014 at 6:34

Although it goes far beyond the question as posed (for which other answers provide a variety of elementary solutions), there is a broader context for this numbertheoretic question.
The setting involves a global field $K$, ring $R$ of $S$integers for a nonempty finite set $S$ of places of $K$ (containing the archimedean places), arbitrary extension field $F$ of $K$ (such as $R = \mathbf{Z}$ and $F = \mathbf{R}$), and Chevalley group $G$ over $R$ (e.g., ${\rm{SL}}_n$, ${\rm{GL}}_n$, ${\rm{Sp}}_{2g}$, ${\rm{PGL}}_n$, etc.).
To formulate a general proposition about normalizers in this setting, we need some more notation. Let $G' = \mathscr{D}(G)$ be the derived group, $Z_G \subset G$ the center, and $G^{\rm{ad}} = G/Z_G = G'/Z_{G'}$ (quotients in the scheme sense that is wellsuited to working over rings such as $R$). For example, if $G = {\rm{GL}}_n$ then $G^{\rm{ad}} = {\rm{PGL}}_n$ and $G' = {\rm{SL}}_n$, but note that ${\rm{PGL}}_n(R)$ is strictly larger than ${\rm{GL}}_n(R)/R^{\times}$ when the class group of $R$ has nontrivial $n$torsion.
We will use the strong approximation theorem over global fields and the Iwasawa decomposition over nonarchimedean local fields (all applied to split connected semisimple groups) to prove:
Proposition: An element $g \in G(F)$ satisfies $gG'(R)g^{1} \subset G'(R)$ if and only if the image of $g$ in $G^{\rm{ad}}(F)$ lies in $G^{\rm{ad}}(R)$.
The sufficiency is obvious due to the natural action of $G^{\rm{ad}}$ on $G$ over $R$, so we need to prove the necessity.
Remark: The condition that $g$ lands in $G^{\rm{ad}}(R)$ is equivalent to saying that $g$ lies in $G(R)Z_G(F)$ when $R$ is a UFD (or when $G$ is semisimple), but it is a strictly weaker condition on $g$ when $G = {\rm{GL}}_n$ and $R$ has nontrivial $n$torsion in its class group. In this respect, the case $R = \mathbf{Z}$ might give the wrong impression as to what happens over more general number fields.
Remark: In the special case $G = {\rm{GL}}_n$, everything that follows can be made entirely explicit in terms of matrices and requires no heavy machinery (e.g., strong approximation for ${\rm{SL}}_n$ can be deduced from strong approximation for adele rings, and the Iwasawa decomposition is just a reformulation of the structure theorem for modules over a PID).
To prove the Proposition, we first show a weaker assertion:
Lemma: The image of $g$ in $G^{\rm{ad}}(F)$ lies in $G^{\rm{ad}}(K)$.
Proof: The key point is to exploit descent theory by considering the automorphism functor of $G_K$ over $K$algebras (such as $F$ and $F \otimes_K F$).
Fix a split maximal $R$torus $T$ in $G$ (provided by the very definition of Chevalley groups, and which can be taken to be the diagonal subgroup when $G = {\rm{GL}}_n$), so we get closed root subgroups $U_a$ in $G'$ over $R$ for every root $a \in \Phi(G,T)$ (described in terms of elementary matrices when $G = {\rm{GL}}_n$). The subset $U_a(R) \subset U_a(K)$ is Zariskidense in $(U_a)_K$ for all $a$ (as $U_a$ is an affine line over $R$), and the subgroups $U_a(K)$ generate a subgroup of $G'(K)$ that is Zariskidense in the connected semisimple $G'_K$. Thus, for any $K$algebra $A$ we can detect equality for a pair of $A$automorphisms of $G'_A$ by checking on the subset $G'(R) \subset G'(A)$.
Let $j_1, j_2:F \rightrightarrows F \otimes_K F$ denote the two natural scalar extensions $x \mapsto x \otimes 1, 1 \otimes x$. Thus, $g$conjugation $c_g$ on $G'_F$ satisfies $(j_1)^{\ast}(c_g) = (j_2)^{\ast}(c_g)$ as automorphisms of the $F \otimes_K F$group $G'_{F \otimes_K F}$ since these automorphisms coincide on $G'(R)$ (as $j_1$ and $j_2$ coincide on $K$, let alone on $R$).
It follows by descent theory that the $F$automorphism $c_g$ of $G_F$ descends to a $K$automorphism of $G'_K$. But we have naturally $G^{\rm{ad}}_K \simeq {G'}^{\rm{ad}}_K$ as $K$groups via the action of $G_K$ on $G'_K$, so an element of $G^{\rm{ad}}(F)$ whose action on $G'_F$ is defined over $K$ must arise from $G^{\rm{ad}}(K)$.
QED Lemma
It suffices to show that any $\overline{g} \in G^{\rm{ad}}(K) = {G'}^{\rm{ad}}(K)$ whose conjugation action on $G'_K$ carries $G'(R)$ into itself lies in the subgroup ${G'}^{\rm{ad}}(R) \subset {G'}^{\rm{ad}}(K)$. In particular, we can now rename $G'$ as $G$ to reduce to the case when $G$ is semisimple (for the purpose of proving our refined claim that has nothing to do with elements of $G'(R)$, but rather with elements of ${G'}^{\rm{ad}}(R)$). Since $G^{\rm{ad}}$ is identified with a closed subgroup scheme of the automorphism scheme ${\rm{Aut}}_{G/R}$ over $R$, a $K$point of $G^{\rm{ad}}$ extends to an $R$point if and only if the associated $K$automorphism of $G_K$ extends to an $R$automorphism of $G$.
Now it suffices to show for any semisimple Chevalley group $G$ over $R$ that any $K$automorphism $f$ of $G_K$ which carries $G(R)$ into itself extends (necessarily uniquely) to an $R$automorphism of $G$.
Letting $q:\widetilde{G} \rightarrow G$ be the simply connected central cover (a central isogeny over $R$, with $\widetilde{G}$ a simply connected Chevalley group), by canonicity there is a unique $R$automorphism $\widetilde{f}$ of $\widetilde{G}$ over $f$. Thus, $\widetilde{f}(\widetilde{G}(R)) \subset \widetilde{G}(K) \cap q^{1}(G(R)) = \widetilde{G}(R)$, where the final equality for the Dedekind $R$ expresses the valuation criterion of properness for the elementary case of the finite map $q$. Hence, we may replace $(G, f)$
with $(\widetilde{G}, \widetilde{f})$ to reduce to the case that the semisimple Chevalley group $G$ over $R$ is simply connected (e.g., ${\rm{SL}}_n$ or ${\rm{Sp}}_{2g}$).
So far we have not used anything about $R$ beyond that it is a Dedekind domain (and the Dedekind property was only used in the final step above with the valuative criterion). It is time to bring in the arithmetic input in order to "localize" out problem (despite the hypothesis in terms of $G(R)$ appearing to be rather "global").
Recall the definition of $R$ as a ring of $S$integers. By the strong approximation theorem for simply connected groups, applied by omitting any place in $S$ (at all of which $G$ is locally split), $G(R)$ has dense image in $G(R_v)$ for all $v \not\in S$; this is classical for $G = {\rm{SL}}_n$ and $R = \mathbf{Z}$. Hence, the closure of $G(R)$ in $G(K_v)$ is $G(R_v)$ for all $v \not\in S$, so the $K_v$automorphism $f_{K_v}$ of $G_{K_v}$ carries $G(R_v)$ into itself for all such $v$. Our task is to show that the $K$automorphism $f$ of $G_K$ extends to an $R$automorphism of $G$ with $R = O_{K,S}$, and for this purpose it suffices to check that $f_{K_v}$ extends to an $R_v$automorphism of $G_{R_v}$ for all $v \not\in S$.
Now our problem is intrinsic to the local situation: by renaming $(K_v, R_v)$ as $(K, R)$ we're given a nonarchimedean local field $K$ with valuation ring $R$ and an automorphism $f$ of $G_K$ for a simply connected Chevalley group $G$ over $R$ such that $f(G(R)) \subset G(R)$ and we claim that $f$ extends to an $R$automorphism of $G$.
Fix a split maximal $R$torus $T$ of $G$ and a Borel $R$subgroup $B$ of $G$ containing $T$. Let $\Phi^+ = \Phi(B,T)$ and let $\Delta$ be the base of $\Phi^+$, so $T^{\rm{ad}} := T/Z_G$ is a split maximal $R$torus of $G^{\rm{ad}}$ whose character group admits $\Delta$ as a $\mathbf{Z}$basis. In particular, a point in $T^{\rm{ad}}(K)$ lies in $T^{\rm{ad}}(R)$ if and only if its image in $K^{\times}$ under each $a \in \Delta$ lies in $R^{\times}$.
For each $a \in \Delta$, let $X_a$ be a basis of the $R$line ${\rm{Lie}}(U_a)$. Since ${\rm{Aut}}_{G/R}/G^{\rm{ad}}$ is identified with the finite constant $R$group of automorphisms preserving the pinning $(B,T, \{X_a\}_{a \in \Delta})$ of $G$ over $R$, we can adjust $f$ by the $K$fiber of a "pinned" $R$automorphism of $G$ so that $f$ arises from $G^{\rm{ad}}(K)$. The Iwasawa decomposition gives $G^{\rm{ad}}(K) = G^{\rm{ad}}(R) T^{\rm{ad}}(K) G^{\rm{ad}}(R)$, so we may assume that $f$ arises from the action of some $t \in T^{\rm{ad}}(K)$. Thus, $f$ preserves every root group $(U_a)_K$ over $K$, inducing the scaling $t^a$ on this affine line over $K$. But this carries $(U_a)(K) \cap G(R) = U_a(R)$ into itself for all $a \in \Phi(G,T)$, which is to say that $t^a$multiplication on $K$ preserves an $R$line for all $a$, so $t^a \in R$ for all $a$. Applying this to $a$ as well, we see that $t^a \in R^{\times}$ for all $a$. By considering $a \in \Delta$, we conclude that $t \in T^{\rm{ad}}(R)$, so $f \in G^{\rm{ad}}(R)$ as desired.
QED Proposition
If you do the computation for the "continued fraction generators", $\tau = \begin{pmatrix} 1 & 0\\0 & 1\end{pmatrix}$ and $\sigma = \begin{pmatrix} 1 & 1\\ 0 & 1\end{pmatrix},$ and write out the equation for $\begin{pmatrix} a & b\\ c & d\end{pmatrix}$ to conjugate $\sigma, \tau$ into integer matrices, you get the conditions that $a^2, b^2, c^2, d^2, ac, bd$ as necessary and sufficient. Which means that the normalizer consists of matrices of the form $\begin{pmatrix} x \sqrt{s} & y \sqrt{t}\\ z \sqrt{s} & w\sqrt{t}\end{pmatrix},$ where $x, y, z, w, s, t$ are integer and $xw\sqrt{st}  zy \sqrt{st} = (xw  zy)\sqrt{st} = 1.$ From this it follows that $\sqrt{st} = 1,$ and thus $s=t=1.$

$\begingroup$ Sorry, Igor, I don't understand why ac = xz\sqrt{st} should be an integer (do you mean it should? there is a part of the formula missing). $\endgroup$ Commented Feb 21, 2014 at 1:11

$\begingroup$ I presume he means $ad$ and $bc$. $\endgroup$ Commented Feb 21, 2014 at 1:17

1$\begingroup$ The center of $GL_2(\mathbb{R})$ should be in every normalizer, so I'm not sure this can be true? $\endgroup$ Commented Feb 21, 2014 at 1:49

$\begingroup$ @SashaKolpakov you are correct, see the edit. $\endgroup$ Commented Feb 21, 2014 at 2:36

2$\begingroup$ The two matrices given in the answer do not generate ${\rm SL}(2,\mathbb{Z}),$ even working up to scalar multiples. $\endgroup$ Commented Feb 21, 2014 at 8:41