This happens iff $G$ has a trivial center and $1\to G\to \mathrm{Aut}(G)\to\mathrm{Out}(G)\to 1$ splits.
Proof: suppose $G$ has nontrivial center $Z$. Let $i$ be the embedding of $Z$ into the center of a perfect group $S$ [note that $S$ can be chosen finite if $G$ is finite]. Consider the "central product": quotient $H$ of $G\times S$ by the central subgroup $\{(z^{-1},i(z)):z\in Z\}$ (i.e., we glue, in $G\times S$, both copies of $Z$ through $i$). We can view $G$ and $S$ as normal subgroups of $H$, intersecting in $Z$. Then the centralizer of $G$ in $H$ is equal to $S$.
We claim that the extension $1\to G\to H\to H/G\to 1$ is not split. Otherwise, write $H=G\rtimes L$. Then the centralizer of $G$ in $H$ is also $Z\rtimes C_L(G)$. So $S=Z\rtimes C_L(G)$, hence $S=Z\times C_L(G)$ since $Z$ is central in $S$. This contradicts that $S$ is perfect.
Hence $G$ has trivial center, and the splitting of the exact sequence $1\to G\to \mathrm{Aut}(G)\to \mathrm{Out}(G)\to 1$ follows by assumption.
Conversely, suppose that $G$ has trivial center and this exact sequence splits. Let $1\to G\to H\to K\to 1$ be an exact sequence. Let $Z$ be the centralizer of $G$ in $H$. Then $N\cap G=1$. So this induces an exact sequence $1\to G\to H/Z\to K/Z\to 1$, which is split, so $H/Z=G\rtimes L/Z$ for some subgroup $L$ of $H$ containing $Z$. Hence $H:G\rtimes L$.
Note also that the characterization within finite groups is the same, by the remark at the beginning of the proof.