Iterated Automorphism Groups Notation: For each group $G$ define:
$Aut^{(0)}(G):=G$
$Aut^{(1)}(G):=Aut(G)$
$\forall n\geq 1~~~Aut^{(n+1)}(G):=Aut(Aut^{(n)}(G))$
Question: Consider $I\subseteq \omega$. Is there a group $G$ such that:
(1) $\forall i\in I~~~~~Aut^{(i)}(G)$ is an Abelian group.
(2) $\forall i\notin I~~~~~Aut^{(i)}(G)$ is a non-Abelian group.
 A: I have a partial answer, which would be a complete answer (that is to say a complete classification of all such sets $I$) if we assume Dickson's conjecture in number theory.
If $G$ is non-Abelian, then $Aut(G)$ is also non-Abelian. Specifically, for two non-commuting elements $a,b \in G$, the inner automorphisms $x \rightarrow a x a^{-1}$ and $x \rightarrow b x b^{-1}$ do not commute.
Hence, $I$ can only be either:


*

*The empty set, by letting $G$ be a non-Abelian group;

*$\mathbb{N}$, by letting $G$ be the trivial group or some 'ancestor' thereof;

*A set of the form $\{ 0, 1, 2, \dots, n \}$ for some $n$.


In the third case, let $P(n)$ denote the proposition that there exists a group $G$ with the property that $Aut^n(G)$ is Abelian but $Aut^{n+1}(G)$ is non-Abelian. We clearly have $P(n) \implies P(n-1)$, by replacing $G$ with $Aut(G)$.
So, we want to show that $P(n)$ is true for arbitrarily large values of $n$.

Lemma 1: If $G$ is a finite non-cyclic Abelian group, then $Aut(G)$ is non-Abelian.
Proof: By the Structure Theorem for finitely-generated modules over a PID, we can express $G$ as a product of cyclic groups of prime power orders. Then, $G$ is non-cyclic if and only if the product contains cyclic groups of orders $p^a$ and $p^b$ for the same $p$. If we can find non-commuting automorphisms of $C_{p^a} \times C_{p^b}$, then we are done, since they would extend to non-commuting automorphisms of $G$.
Without loss of generality assume $a \geq b$. and consider these two automorphisms:


*

*The automorphism $f : C_{p^a} \times C_{p^b} \rightarrow C_{p^a} \times C_{p^b}$ sending $(x,y)$ to $(x,y+x)$.

*The automorphism $g : C_{p^a} \times C_{p^b} \rightarrow C_{p^a} \times C_{p^b}$ sending $(x,y)$ to $(x+p^{a-b}y,y)$.


These do not commute. In particular, $f(g(1,0)) = (1,1)$ whereas $g(f(1,0)) = (1+p^{a-b},1)$.

Lemma 2: If $G$ is a cyclic group that does not eventually become the trivial group under repeated iteration of $Aut$, then $G$ will eventually become non-Abelian.
Proof: Induction on the order of $G$. If $Aut(G)$ is non-cyclic, then $Aut(Aut(G))$ is non-Abelian by the previous lemma. If $Aut(G)$ is cyclic and $G$ is not the trivial group, then $Aut(G)$ is smaller than $G$ and we are done by induction.

We know precisely which cyclic groups lead to the trivial group (http://oeis.org/A117729). Any other cyclic group will thus eventually lead to a non-Abelian group.
Assuming Dickson's conjecture, we can find arbitrarily long Cunningham chains of the first kind. These are sequences $(p_1, \dots, p_m)$ of primes where $p_{i+1} = 2p_i + 1$. Note that under the iterated automorphism operator, we get a nice long sequence of cyclic groups:
$C_{2 p_m} \rightarrow C_{2 p_{m-1}} \rightarrow \dots \rightarrow C_{2 p_1}$
Also, if $m \geq 6$ then these groups do not belong to A117729, so must eventually lead to a non-Abelian group by Lemma 2.
Consequently, we establish proposition $P(n)$ for some $n \geq m$, and (assuming Dickson's conjecture) this gives us $P(n)$ for all $n \in \mathbb{N}$.
It's not hard to show a partial converse: if we restrict ourselves to finite groups, $P(n)$ is true for all $n$ if and only if there exist arbitrarily long Cunningham chains.

Without Dickson's conjecture, how far can we go? Well, starting with $G$ being the cyclic group of order $361736822347711983585853438$ (twice the largest element of the longest known Cunningham chain), it takes $19$ iterations to reach a non-Abelian group. Hence, $P(n)$ is true for $n \leq 18$.
