This happens iff $G$ has a trivial center and $1\to G\to \mathrm{Aut}(G)\to\mathrm{Out}(G)\to 1$ splits.
Proof: suppose $G$ has nontrivial center $Z$. Let $i$ be the embedding of $Z$ into the center ($*$) of a perfect group $S$ [note that $S$ can be chosen finite if $G$ is finite]. Consider the "central product": quotient $H$ of $G\times S$ by the central subgroup $\{(z^{-1},i(z)):z\in Z\}$ (i.e., we glue, in $G\times S$, both copies of $Z$ through $i$). We can view $G$ and $S$ as normal subgroups of $H$, intersecting in $Z$. Then the centralizer of $G$ in $H$ is equal to $S$.
We claim that the extension $1\to G\to H\to H/G\to 1$ is not split. Otherwise, write $H=G\rtimes L$. Then the centralizer of $G$ in $H$ is also $Z\rtimes C_L(G)$. So $S=Z\rtimes C_L(G)$, hence $S=Z\times C_L(G)$ since $Z$ is central in $S$. This contradicts that $S$ is perfect.
Hence $G$ has trivial center, and the splitting of the exact sequence $1\to G\to \mathrm{Aut}(G)\to \mathrm{Out}(G)\to 1$ follows by assumption.
Conversely, suppose that $G$ has trivial center and this exact sequence splits. Let $1\to G\to H\to K\to 1$ be an exact sequence. Let $Z$ be the centralizer of $G$ in $H$. Then $N\cap G=1$. So this induces an exact sequence $1\to G\to H/Z\to K/Z\to 1$, which is split, so $H/Z=G\rtimes L/Z$ for some subgroup $L$ of $H$ containing $Z$. Hence $H:G\rtimes L$. $\Box$
Note also that the characterization within finite groups is the same, by the remark at the beginning of the proof.
($*$) this is correct but a weaker statement is enough: Every abelian group $A$ is contained in the intersection of the center and derived subgroup of a group $S$.
Proof: let $u$ be the automorphism $(x,y)\mapsto (x+y,y)$ of $A^2$. Note that $(x,y)-u(x,y)=(y,0)$. Then the semidirect product $A^2\rtimes \langle u\rangle$ [which is finite if $A$ is finite] contains $Z\times\{0\}$ as central subgroup, consisting of commutators since writing $v=(x,y)$ and $w=(y,0)$, the previous equality reads as $w=vuv^{-1}u^{-1}$ in the semidirect product. $\Box$
