The answer to your question is no for finite groups. That is there are no other finite examples that satisfy your condition.

A finite $p$-group $P$ with $\lvert P\rvert>p$ has an outer automorphism of $p$-power order. That is a well-known result of Gaschütz for nonabelian $P$, and is easily checked for abelian $P$.

So all Sylow $p$-subgroups of a finite group $G$ satisfying your condition must be cyclic of order $p$. Then, by repeated application of Burnside's Transfer Theorem to the primes dividing $\lvert G\rvert$ in increasing order, we find that $G$ has a normal Sylow $p$-subgroup $P$, where $p$ is the largest prime dividing $\lvert G\rvert$.

Your condition implies that $N_G(P)/C_G(P)=G/C_G(P)$ is isomorphic to $\operatorname{Aut}(P)$, which is cyclic of order $p-1$.

Now, if $p>3$, then (since $p-1$ is square-free), there is some prime $q$ with $2<q<p$ and $q \mid p-1$, and there exists $N \lhd G$ with $C_G(P) < N$ and $\lvert G/N\rvert=q$, so $G=NQ$ with $Q \in \operatorname{Syl}_q(G)$. But now $N_G(Q) = N_N(Q)Q = C_G(Q)$, so the automorphisms of $Q$ cannot be induced by elements of $N_G(Q)$, contrary to assumption.

Hence $p \le 3$, and $G=\operatorname{Sym}(n)$ with $n=1$, $2$ or $3$.

6more comments