What is the group of outer automorphisms of $SL_n(\mathbb{Z})$. I wanted to understand semidirect products of the form $SL_n(\mathbb{Z})\rtimes_\varphi \mathbb{Z}$ and its isomorphism type depends only on $[\varphi]\in Out(SL_n(\mathbb{Z})$. There is always the conjugate inverse, which is clearly not an inner automorphism, as it doesn't preserve the minimal polynomial of the matrix (at least for $n\ge 3$). Are there any other outer automorphisms ?
4 Answers
As I suggested in my short comment, this kind of question has been around for a long time and has led to a vast amount of literature. It probably starts with work over fields by Schreier and van der Waerden in the 1920s, then considerable work by Dieudonne, O'Meara, and many others. Two indications about what's out there are a survey by O'Meara The integral classical groups and their automorphisms (1969) and a short paper by me dealing from the algebraic group viewpoint with groups like $SL_n(\mathbb{Q})$, On the automorphisms of infinite Chevalley groups, Canad. J. Math. 21 1969 908–911, the latter probably available online by now.
These are listed on MathSciNet, but I couldn't display the links there for some reason. I'll try to suggest more focused literature on your question when I get time today.
P.S. Ed Formanek has pointed to a basic early paper by Hua and Reiner here. Like many other papers on related automorphism groups, the emphasis is on identification of special types of automorphisms which suffice to generate the whole group: inner, "field" or "ring" (as in complex conjugation and the like), "graph" (as in transposeinverse map for special linear groups), "diagonal" (as in the use of conjugation by diagonal matrices not of determinant 1 to produce automorphisms of a subgroup). Naturally the ring or field automorphisms play no role over the rational integers or rational numbers.
Conjugating by an element $Q \in GL_n(\mathbf{Z})$ is usually not inner, furthermore the graph automorphism $A \mapsto (A^T)^{1} $ is not inner.
The phrase to google for is "automorphisms of Chevalley groups over rings" which in particular turns up recent work of Bunina.
By invoking Margulis superrigidity, for $n \geq 3$, any automorphism of $SL_n(\mathbf Z)$ extends to a rational automorphism of $SL_n(\mathbf Q)$. To show that the abovementioned examples then are the only ones, one calculates the stabilizer of $SL_n(\mathbf Z)$ in $GL_n(\mathbf Q)$.

$\begingroup$ I didn't get the last part. Okay any auromorphism extends to a automorphism of $SL_n(\mathbb{Q})$. But then some knowledge of the automorphisms of $SL_n(\mathbb{Q})$ must enter, right? What is this knowledge exactly ? $\endgroup$ Mar 3, 2011 at 11:00

1$\begingroup$ Any rational automorphism of $SL_n(\mathbf C)$ preserves the root datum associated to this semisimple algebraic group, see e.g. Springer's book on linear algebraic groups. In more downtoearth terms, up to an inner automorphism, the diagonal matrices will be preserved, and the subgroups $e_{ij}$ of elementary matrices will be permuted. This permutation, if nontrivial, then can be shown to coincide with $A \mapsto (A^{1})^T$. Looking at the induced map $e_{ij}(s) \mapsto e_{ij}(s)$ it follows that the automorphism must be in $GL_n(\mathbf C)$. Then there are rationality questions... $\endgroup$– GuntramMar 3, 2011 at 11:40

$\begingroup$ @Guntram: As I mentioned to Keivan Karai, these sophisticated approaches including appeal to work of Margulis are not at all needed for the elementary study of the various automorphism groups here. The older concrete methods are fairly elementary and hard to improve on. $\endgroup$ Mar 4, 2011 at 14:42
One possible approach (perhaps not the most elementary one) is to use Margulis' Superrigidity theorem. $\Gamma=SL_n({\mathbb Z})$ is a lattice in the simple Lie group $SL_n({\mathbb R})$ which has rank $n1 \ge 2$ when $n \ge 3$. Now, if $\phi: \Gamma \to SL_n({\mathbb R}$ is a homomorphism such that $\phi(\Gamma)$ is Zariski dense, the abovementioned theorem guarantees that $\phi$ extends to a {\it rational} homomorphism $\phi': SL_n({\mathbb R}) \to SL_n({\mathbb R})$. This condition is automatically satisfied here, so the problem boils down to characterizing the rational automorphism of $SL_n({\mathbb R})$. This is easy, because you can show that conjugations with matrices in $GL_n(\mathbb R)$ form a subgroup of index 2 there, and generate the full group with with $x \mapsto ^tx^{1}$. You have to check which one of these will stabilize $SL_n({\mathbb Z})$. It is certainly in $SL_n({\mathbb Q})$ which is the commensurability subgroup. I have not checked this, but I guess that the normalizer here is exactly $SL_n({\mathbb Z})$.
The case of $SL_2(\mathbb Z)$ is probably more complicated. Superrigidity is certainly false, at least for some subgroups of finite index in $SL_2(\mathbb Z)$, since $SL_2(\mathbb Z)$ has free subgroups of finite index with a lot of automorphisms. If they do extend to $SL_2$ (I am not sure if this is the case) they you get an abundance of automorphisms that are not algebraic, and perhaps you cannot easily classify them.

$\begingroup$ You forgot about diagonal automorphisms of $SL_n(\mathbf R)$, i.e. those which are induced by conjugating with a diagonal matrix in $GL_n(\mathbf R)$. $\endgroup$– GuntramMar 3, 2011 at 12:34

$\begingroup$ In the broader setting of discrete subgroups of Lie groups including arithmetic groups, the methods of Margulis are extremely powerful. But not so much theory is needed to compute the automorphisms of some familiar linear groups. $\endgroup$ Mar 3, 2011 at 19:50

$\begingroup$ @Jim Humphreys: I totally agree, and that's why I started with "a possible approach". I would be quite pleased to see the ideas behind other methods, which are perhaps more suitable for this problem. $\endgroup$ Mar 3, 2011 at 21:23
See L. K. Hua and I. Reiner, "Automorphisms of the unimodular group", Trans. Amer. Math. Soc. 71 (1951), 331348.

$\begingroup$ This is probably the earliest published treatment, though I had the impression that I'd seen a later textbook version. The article itself is freely available from the AMS Transactions website (like all other old AMS journal articles). $\endgroup$ Mar 3, 2011 at 19:21
$n=2$
separately. In any case, both the question and its answer are fairly old, so it's mostly a matter of locating the most useful and accessible reference. On the other hand, the question is part of a much larger recent study of arithmetic groups and their automorphisms over rings of algebraic integers, etc. $\endgroup$