You have noted (accurately) that $G(p) =G$ unless $G$ has a cyclic Sylow $p$-subgroup. However, it is also clear that when $G$ has a cyclic Sylow $p$-subgroup and $G(p) \neq G$$G(p) \neq G,$ the group $G$ has a normal $p$-complement. For, otherwise, we have by Burnside's transfer theorem, there is a $p$-regular element $x \in N_{G}(P) \backslash C_{G}(P)$. Now we have $P = [P,x] \times C_{P}(x).$ Since $P$ is cyclic and $P \neq C_{P}(x),$ we must have $P = [P,x]$ and $C_{P}(x) = 1.$ But then $P \leq G^{\prime},$ a contradiction, since $[G:G(p)] =p$ and $G(p)$ is clearly a normal subgroup when it is a subgroup.
On the other hand, if $G$ has a normal $p$-complement and cyclic non-trivial Sylow $p$-subgroup, thethen clearly $G$ has a normal subgroup of index $p$, and that that subgroup is indeed $G(p)$.
The conclusion (when $|G|$ has order divisible by $p,$ is that $G(p)$ is a subgroup of $G$ if and only if one of the following cases occur:
$G(p) = G$, or, $G$ has a normal $p$-complement and a cyclic Sylow $p$-subgroup. Notice
Notice that when $p =2$, every finite group with a cyclic Sylow $2$-subgroup is known to have a normal $2$-complement, so the condition given for $p$ odd to guarantee that $[G:G(p)] = p$ is somewhat analogous to what you observed when $p =2$. It is indeed sufficient that $G$ should have a cyclic Sylow $p$-subgroup when $p$ is the smallest prime divisor of $|G|$ to ensure that $[G:G(p)] = p$. It is not, however, necessary.