It is probably best to concentrate on groups $G$ with $Z(G) =1$. For example, a finite group $G$ which is a direct product of non-Abelian simple groups is characterized by its automorphism group. For $G \cong {\rm Inn}(G)$ is isomorphic to a normal subgroup of $A = {\rm Aut}(G)$. I claim that $G \cong E(A)$ (which is the central product of all components-that is, quasisimple subnormal subgroups- of $A$). Since $G \cong {\rm Inn}(G)$ in a natural manner we may regard $G$ as a (normal) subgroup of $A$, and then we certainly have $G \leq E(A)$. Now suppose that $A$ has a component $L$  which is not contained in $G$. Then $[L,G] = 1$ by standard properties of components (just the three subgroups Lemma is needed). But $L$ is a subgroup of ${\rm Aut}(G)$, a contradiction. Hence $G$ is isomorphic to $E(A)$, which is a uniquely determined characteristic subgroup of $A$.