As suggested by Igor Rivin's argument (and following lines of argument suggested by Frobenius in the late 19th/early 20th century), it is the case that if we put the operator norm (with respect to Euclidean norm on vectors) on $M_{n}(\mathbb{C})$ and consider a finite subgroup $G$ of $U_{n}(\mathbb{C})$ such that $\|g^{2}-e \| < \frac{1}{6}$$\|g^{2}-e \| < 1$ for all $g \in G$, then $G$ is Abelian.
We may supposeLater edit: I realise that $G$ is irreducible, and primitive ( ie the given representationthere is not induced from a representation ofmuch simpler argument than my original one, with a proper subgroup)sharper bound. We may and do, assumeNote that $G$$M$ is not nilpotent, since all irreducible complex representationsa matrix of nilpotent groups are induced fromfinite order in $1$-dimensional representations, and the result is clear if${\rm GL}(n,\mathbb{C})$ with $n = 1$. Since$\| I - M^{j} \| < 1$ for all $G$ is primitive$j$, every Abelian normal subgroup ofthen for any eigenvalue $G$ consists$\lambda$ of scalar matrices$M$, we have $|1-\lambda^{j}| < 1$ for all $j$, so is central. We will obtain a contradiction by showing that $G^{\prime}$ is Abelian ($\lambda^{j}$ has strictly positive real part for then $G^{\prime} \leq Z(G)$ andall $G$ is nilpotent)$j$.
Frobenius showed But (actually with a different matrix norm, though$1$ is the argument can be modified to workonly complex root of unity with the operator norm) that the elements $x \in G$ which satisfy $\| I - x \| < \frac{1}{2}$ generate an Abelian normal subgroup of $G$. We showproperty that $\| ab - ba \| < \frac{1}{2}$ for all $a,b \in G$, so thatof its powers have positive real part $\|I - a^{-1}b^{-1}ab \| < \frac{1}{2}$ for( since the sum of all powers of any other root of unity is $a,b \in G$, and hence$0$). Hence $G^{\prime}$ is Abelian$M = I$.
NowHence if $\|ab - ba \| \leq \|ab - ab^{-1}\| + \|ab^{-1} - a^{-1}b^{-1} \| + \|a^{-1}b^{-1} -ba \|$ =$\|I -g^{2} \| < 1$ for all $\|b - b^{-1}\| + \|a - a^{-1} \| + \|(ba)^{-1} -ba \|$ =$g \in G$, where $\|b^{2} - I\| + \|a^{2} - I \| + \|(ba)^{2}-I \|$ <$G$ is a finite subgroup of $\frac{1}{2}$. Thus${\rm U}_{n}(\mathbb{C})$, then $\| I - a^{-1}b^{-1}ab \| < \frac{1}{2}$$g^{2} = I$ for all $a,b \in G$$g \in G$, so that $G^{\prime}$$G$ is Abelian, the required contradiction.
The second part of the answer is somewhat tangential to the original question, and follows the direction suggested by Sean Eberhard's comment.
For a finite group $G$, it is the case that if more than $\sqrt{\frac{5}{8}} |G|$ elements $x \in G$ have $x^{2} = e$, then $G$ is Abelian. The dihedral group of order $8$ ( I mean the one with $8$ elements) - and direct products of it with elementary Abelian $2$-groups as large as you like-show that this can't be improved much as a general bound, since a dihedral group $D$ of order $8$ contains $6$ elements which square to the identity and $ 6 < \sqrt{\frac{5}{8}} |D| <7$ in that case.
This is because (as noted in the paper linked to in Sean Eberhard's comment, and also previously noted by Brauer and Fowler), the count of solutions to $x^{2} = e$ given using the Frobenius-Schur indicator leads easily to $\sqrt{\frac{5}{8}} |G| < \sqrt{k(G)}\sqrt{|G|}$ in the case under consideration, where $k(G)$ is the number of conjugacy classes of $G$. Hence $\frac{k(G)}{|G|} > \frac{5}{8}$, so the probability that two elements of $G$ commute is greater than $\frac{5}{8}$, in which case $G$ is Abelian by a Theorem of W.Gustafson. If preferred, this can be seen directly using character theory- in general, if $G$ has $k$ conjugacy classes, we have $[G:G^{\prime}] + 4(k - [G:G^{\prime}]) \leq |G|$ by the orthogonality relations. Hence $\frac{k}{|G|} \leq \frac{1}{4} + \frac{3}{4|G^{\prime}|} \leq \frac{5}{8}$ if $G^{\prime} \neq 1$, ie if $G$ is non-Abelian.