Skip to main content
3 of 3
completed answer
YCor
  • 63.9k
  • 5
  • 187
  • 286

It's an exercise to check it coincides with $\mathrm{Aut}(S_6)$ which has $S_6$ as subgroup of index 2.

Here are the steps. First, for arbitrary $n\ge 6$, consider the graph of transpositions. So this is the set $X_n$ of the $n(n-1)/2$ unordered pairs, with an edge between two whenever they are disjoint.

On $X_n$, consider the set $Y_n$ all unordered pairs of non-joined vertices (hence of the form $\{\{a,b\},\{a,c\}$ for $a,b,c$ pairwise distinct. There's a canonical map $\phi:Y_n\to\{1,\dots,n\}$ given by $\phi(\{\{a,b\},\{a,c\})=a$; we wish to show it's equivariant for the group action.

Link two elements of $Y_n$ if they are disjoint and contained no pair of joined vertices. Then any two linked elements of $Y_n$ have the same image by $\phi$ (check! this uses the disjointness assumption). Next consider the equivalence relation on $Y_n$ generated by this equivalence relation. The only 2-element subsets in $Y_n$ with same image by $\phi$ but not linked have the form, up to permutation, $\{\{12,13\},\{12,14\}\}$. But one indeed have $\{12,13\}-\{15,16\}-\{12,14\}$. So for $n\ge 6$, the equivalence relation on $Y_n$ generated by being linked consists of being in the same fibers of $\phi$, and hence the automorphism group of $X_n$ acts on $\{1,\dots,n\}$. This action is easily seen to be faithful. Since the permutation group $S_n$ already acts, this shows that the automorphism group of $X_n$ is $S_n$.

Next in $S_n$ for $n\ge 7$, the transpositions are the only elements whose centralizer has order $2(n-2)!$, i.e., form the only conjugacy class of order $n(n-1)/2$, and hence $\mathrm{Aut}(\mathrm{Comm}(S_n))$ stabilizes its subgraph $X_n$. Again a little argument shows that this action is faithful (i.e. an automorphism of $\mathrm{Comm}(S_n)$ fixing pointwise the transpositions is the identity; also for $n=6$.

For $n=6$ on the other hand, there are 2 conjugacy classes of order 15: transpositions and triple transpositions. They are switched by non-inner automorphisms. Hence the previous argument applies to the subgroup of index 2 of $\mathrm{Aut}(\mathrm{Comm}(S_6))$ stabilizing $S_6$ and the result follows.


Update: my answer above was incomplete since it does not address double transpositions; let me now answer the question fully; this will actually make use of the above answer about the graph of transpositions!

Let us consider the graph $Z_n$ of double transpositions. Let me stick to $n=6$. I'll denote the permutation $(ab)(cd)$ as $(ab|cd)$ to avoid too many parentheses/commas.

In $Z_6$, there are triangles defined as follows:

  1. Triangles of type I: $S_6$-permutes of the triangle $T_1=\{(12|34),(13|24),(14|23)\}$.
  2. Triangles of type II: $S_6$-permutes of the triangles $T_2=\{(12|34),(12|56),(34|56)\}$.

It is straightforward that every triangle is of type I or type II, that the $S_6$-action preserves types of triangles, and that elements of $\mathrm{Aut}(S_6)\smallsetminus S_6$ exchange the two types. Consider the graph $W_6$ whose vertices are triangles in $Z_6$ and such that there is an edge between two triangles whenever they have a common vertex. For instance, there is an edge between $T_1$ and $T_2$ since they have the common vertex $(12|34)$. It is easy to check that $W_6$ is connected. Vertices of $W_6$ thus have type I or II, and we see that no vertices of the same type are joined. So $W_6$ is bipartite. Thus, for vertices of $W_6$, the relation "to have the same type" (which is not defined intrinsically) is indeed intrinsic, as it is equivalent to be at even distance. In particular, the isometry group $G$ of $W_6$ preserves the partition by types. Let $G'$ be its index 2 subgroup of type-preserving isometries. Define $W'_6$ as the graph consisting of vertices of type I in $W_6$, linked whenever they have distance at most 2 in $W_6$. We see that $W'_6$ is canonically isomorphic to the transposition graph! The isomorphism is given by mapping the triangle $T_1=\{(12|34),(13|24),(14|23)\}$ to $(56)$, etc.

By the previous result on $X_6$, we deduce that $G'=S_6$. It follows that $G$ is reduced to $\mathrm{Aut}(S_6)$ (since clearly an isometry of $W_6$ that is identity on vertices of type I has to be the identity). To conclude that $\mathrm{Aut}(Z_6)$ is reduced to $\mathrm{Aut}(S_6)$, it is enough to show that the canonical homomorphism $\mathrm{Aut}(Z_6)\to\mathrm{Aut}(W_6)$ is injective. This is also immediate: for $f$ in the kernel, any double transposition belongs to exactly two triangles, i.e., two unordered triples of linked vertices, each of which is $f$-invariant, and hence is fixed by $f$. So we have proved that $\mathrm{Aut}(Z_6)$ is reduced to $\mathrm{Aut}(S_6)$, answering the original question.

YCor
  • 63.9k
  • 5
  • 187
  • 286