Given a finite group G, we denote by Aut(G) the group automorphisms of G . Which finite groups G can be characterized by the group Aut(G), i.e. Aut(H)≅Aut(G) implies H≅G?

$\begingroup$ You need to be more precise about what you mean by $Aut(H)=Aut(G)$. Do you mean that $Aut(H)$ is isomorphic (as an abstract group) to $Aut(G)$? $\endgroup$– verretCommented Dec 23, 2015 at 9:07

7$\begingroup$ Carmichael's conjecture states that the equation $\varphi(x)=n$ (where $n$ is given) never has a unique solution. This amounts to saying that the finite cyclic group $\mathbb{Z}/n\mathbb{Z}$ is never characterized by the order of its automorphism group. $\endgroup$– Alain ValetteCommented Dec 23, 2015 at 9:08

$\begingroup$ Sorry, Aut(H)≅Aut(G) implies H≅G? $\endgroup$– R. ShhaiedCommented Dec 23, 2015 at 10:09
1 Answer
It is probably best to concentrate on groups $G$ with $Z(G) =1$. For example, a finite group $G$ which is a direct product of nonAbelian simple groups is determined by its automorphism group. For $G \cong {\rm Inn}(G)$ is isomorphic to a normal subgroup of $A = {\rm Aut}(G)$. I claim that $G \cong E(A)$ (which is the central product of all componentsthat is, quasisimple subnormal subgroups of $A$). Since $G \cong {\rm Inn}(G)$ in a natural manner we may regard $G$ as a (normal) subgroup of $A$, and then we certainly have $G \leq E(A)$. Now suppose that $A$ has a component $L$ which is not contained in $G$. Then $[L,G] = 1$ by standard properties of components (just the three subgroups Lemma is needed). But $L$ is a subgroup of ${\rm Aut}(G)$, a contradiction. Hence $G$ is isomorphic to $E(A)$, which is a uniquely determined characteristic subgroup of $A$.
Later edit: To be strict, I realise that this does not answer the question as asked. If $G$ is as in my answer, and two or more of the simple direct factors of $G$ are isomorphic, or if the simple direct factors of $G$ are pairwise nonisomorphic but at least one direct factor has nontrivial outer automorphisms, the above argument shows that ${\rm Aut}(A) \cong A = {\rm Aut}(G)$, while $G \not \cong A$, as $G$ has nontrivial outer automorphisms in those cases.
Hence to answer the question as asked, we need to take $G$ to be a direct product of pairwise nonisomorphic nonAbelian simple groups, each of which has trivial outer automorphism group.