This answer is written after taking into account Dave's answer https://mathoverflow.net/a/454042/14094 and the comments therein.
Here's the Baer-Suzuki theorem
If any two elements of a conjugacy class $C$ of a finite group generate a nilpotent subgroup, then all elements of the
conjugacy class $C$ are contained in a nilpotent subgroup.
Also recall that in a nilpotent group, for each $p$ the set of elements of $p$-power order forms a subgroup (i.e., there's a unique $p$-Sylow).
Say that a group has Property $X(p)$ if it is generated by a conjugacy-invariant subset $S$ such that every element in $S$, and also the product of any two conjugate elements in $S$, has $p$-power order.
Note that Property $X(p)$ passes to quotients (unlike, a priori, the initial property of OP).
Claim: every finite group with Property $X(p)$ is a $p$-group.
Let $G$ be a counterexample of minimal cardinal, and $S$ as above. Let $T$ be a conjugacy class in $S$. By the Baer-Suzuki theorem, $T$ generates a nilpotent subgroup $N$, necessary a normal $p$-subgroup. So $G/N$ is also not a $p$-group, and inherits Property $X(p)$, contradiction.
Of course the property considered by OP (which requires that $S$ consists of those elements of order 2) implies $X(2)$. Hence, assuming finiteness, it forces the group to be a 2-group. However, the argument above doesn't directly work with OP's property, because a priori this property doesn't pass to quotients (due to the fact that passing to a quotient produces new elements of order 2).